Add pagination filter features to Thingifier

[eviltester]

Add as API configuration to switch on and off from collections
default is 'on' paging is allowed

Add ?limit and ?offset to configure paging

e.g. GET /api/collection?limit=5&offset=7

configurable default limit and max limit in the api config

config default limit to 10 and max 20

param default offset is 0

When enabled surface in the Auto generated API docs.

When combined with sort the sort should be honoured.

use _ to avoid conflict with current filter approach which uses field names

?_limit=20&_offset=0
?_sortBy=+id&_limit=20&_offset=20

[eviltester]

Thingifier Pagination Plan

Summary

Add collection pagination for Thingifier API reads using reserved query params _limit and _offset. Paging is enabled by default, uses zero-based offsets, applies after filtering and sorting, and is documented only when enabled.

The current branch is codex/issue-90-pagination, based on current origin/master.

Key Changes

• Extend API param config with:
  • willAllowPagingThroughUrlParams() defaulting to true
  • setAllowPagingThroughUrlParams(boolean)
  • defaultPagingLimit() / setDefaultPagingLimit(int) defaulting to 10
  • maxPagingLimit() / setMaxPagingLimit(int) defaulting to 20
• Use only underscored public params:
  • _limit: max number of collection items to return
  • _offset: zero-based number of sorted/filtered items to skip
  • Do not add support for bare limit or offset; those remain ordinary field-filter names for compatibility.
• Apply paging to collection and relationship-collection reads for GET and QUERY; do not apply to single-instance reads.
• Effective API behavior:
  • If paging is enabled, collection reads use _limit or configured default limit, capped at configured max.
  • _offset defaults to 0.
  • _limit=0 returns an empty collection.
  • _offset past the end returns an empty collection.
  • Invalid or negative _limit/_offset returns 400.
  • If paging is disabled, _limit/_offset are ignored, not treated as filters, and not shown in docs.
• Keep sorting precedence:
  • filter first
  • sort second
  • paginate last
  • Example: ?_sortBy=+id&_limit=20&_offset=20 returns the second 20-row page after ascending sort.

Implementation Shape

• Add a small core pagination parser/helper for reserved _limit and _offset, similar in spirit to existing _sortBy handling.
• Ensure _limit, _offset, and _sortBy are treated as reserved query controls and excluded from field filtering in both in-memory and SQLite query paths.
• In the API handlers, build effective collection query params from config before executing the read:
  • inject default/capped _limit
  • inject default _offset=0
  • strip paging controls when paging is disabled
  • keep filtering-disabled enforcement focused on actual field filters, not pagination controls
• In the in-memory repository, slice the filtered and sorted list.
• In SQLite, append LIMIT/OFFSET after ORDER BY.
• Update generated docs:
  • HTML API docs mention _limit and _offset globally and per collection endpoint when paging is enabled.
  • OpenAPI/Swagger includes _limit and _offset query parameters for collection and relationship-collection operations when enabled.

Test Plan

• Core query/repository tests:
  • explicit _limit returns the requested number of items
  • explicit _offset skips zero-based rows
  • filtering + sorting + pagination happens in that order
  • relationship collection pagination works
  • in-memory and SQLite stores return the same paged results
• API handler tests:
  • default collection response is capped at 10
  • _limit above max is capped at 20
  • _offset=0 includes the first item
  • invalid and negative paging params return 400
  • disabling paging returns unpaged collections and ignores _limit/_offset
  • paging works even when field filtering is disabled, because it is its own config feature
• Documentation tests:
  • HTML docs include _limit/_offset when paging is enabled
  • HTML docs omit them when paging is disabled
  • Swagger/OpenAPI includes integer _limit and _offset params when enabled
  • Swagger/OpenAPI omits them when disabled

Assumptions

• _offset is zero-based, defaulting to 0.
• No response wrapper, total count, next/previous links, or pagination headers are added in this issue.
• Pagination is an API query-control feature, independent from field filtering.
• Unprefixed limit and offset are intentionally not implemented.

[eviltester]

Implementation update

Pagination for collection reads has been implemented on branch codex/issue-90-pagination.

Completed

• Added API parameter configuration for paging:
  • willAllowPagingThroughUrlParams() defaulting to true
  • setAllowPagingThroughUrlParams(boolean)
  • defaultPagingLimit() / setDefaultPagingLimit(int) defaulting to 10
  • maxPagingLimit() / setMaxPagingLimit(int) defaulting to 20
• Added reserved paging query controls:
  • _limit
  • _offset
• Kept bare limit and offset as ordinary field-filter names for compatibility.
• Added core pagination parsing and list pagination helpers.
• Updated query filtering so _sortBy, _limit, and _offset are reserved controls and are excluded from normal field filtering.
• Updated GET and QUERY handlers to build effective query params from API config before executing collection reads.
• Applied pagination only to collection and relationship-collection reads, not single-instance reads.
• Preserved precedence:
  • filter first
  • sort second
  • paginate last
• Implemented in-memory paging by slicing the filtered and sorted collection.
• Implemented SQLite collection paging with LIMIT / OFFSET after ORDER BY.
• Implemented relationship-collection paging after collecting, filtering, and sorting related instances.
• Added validation so invalid or negative _limit / _offset returns 400 when paging is enabled.
• Added support for _limit=0 returning an empty collection and _offset past the end returning an empty collection.
• When paging is disabled, _limit / _offset are ignored and not treated as field filters.
• Updated generated HTML API docs to mention _limit and _offset only when paging is enabled.
• Updated OpenAPI/Swagger generation to include integer _limit and _offset query parameters for collection and relationship-collection operations only when paging is enabled.

Tests added or updated

• Core repository/query tests for explicit limits, offsets, offset past end, reserved query params, and bare limit / offset compatibility.
• Store contract coverage for both in-memory and SQLite behavior.
• API handler tests for default limit, max cap, zero offset, zero limit, invalid params, disabled paging, and paging when field filtering is disabled.
• QUERY handler coverage for filter + sort + pagination order.
• HTML documentation tests for enabled and disabled paging docs.
• Swagger/OpenAPI tests for enabled and disabled paging parameters.
• API config tests for defaults and profile copying.

Verification

• Full mvn test passed across all 14 modules after freeing port 4567.
• Local Maven install completed with:
  • mvn install "-Dcheckstyle.skip=true" "-DskipITs=true"

Notes on install: plain mvn install is currently blocked by unrelated existing Checkstyle architecture violations and CRUD UI integration-test failures, so those gates were skipped for the local artifact install. The full standard test suite passed.