Skip to content

Commit c67a3de

Browse files
authored
fix: report list routes return all rows up to a uniform cap (#465)
* fix(reporting): report list routes return all rows up to a uniform cap, refuse bad bounds Report list routes in all five ports now return every row up to a generator-configurable cap (default 10000) instead of a silent 50-row page. Above the cap, or with an explicit limit above it, the route answers 400 report_too_large with maxRows. limit<=0, negative offset and non-integer bounds are refused with the existing 400 envelope (C# used to ignore them). Pinned by new api-contract report scenarios. * no-mistakes(review): Fix TS report limit error text; rename Java test * no-mistakes(test): Fix report cap test filterAllowlist type to typecheck * no-mistakes(document): Docs checked against report row cap change. CHANGELOG, api-contract.md and reporting.md already updated by the change and match code (cap 10,000, reportMaxRows option names, report_too_large and pagination.invalid_value envelopes, 30-scenario count). Fixed one formatting nit: missing blank line between Fixed entries in CHANGELOG [Unreleased]. docs/superpowers/plans/2026-10-04-fr-044-plan-3 still describes the old 50-row default, but it is a dated plan record, so left as is.
1 parent 1cc1b1e commit c67a3de

49 files changed

Lines changed: 1175 additions & 87 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,19 @@ here.**
1212

1313
### Changed
1414

15+
- **A report list route answers every row up to one cap, and is refused above it, in all five
16+
ports.** With no `limit` a served `object.report` returns all of its rows; one row past the
17+
cap, or an explicit `limit` above it, is `400 { "error": "report_too_large", "maxRows": N }`,
18+
before any body is written. The cap is 10,000 by default and a generator option:
19+
`routesFile({ reportMaxRows })` / `routesFileHono` (TypeScript), `RoutesGenerator.ReportMaxRows`
20+
(C#), the `reportMaxRows` generator arg (Java, Kotlin), `router_generator(report_max_rows=)`
21+
(Python). **This changes served output in Java, Kotlin and Python**, whose report lists used to
22+
stop at 50 rows with no limit; and in every port a report longer than the cap that used to answer
23+
now answers `report_too_large`. Entity and projection lists keep their paging. A report page
24+
bound that is not usable (`limit` below 1, a negative `offset`, not an integer) is
25+
`400 pagination.invalid_value` in every port. Gated by the new `ReadingLog` report and the
26+
`cap-all-rows`, `cap-refused` and `bad-page-bound` scenarios of the api-contract `report/` corpus.
27+
1528
- **Requirements guidance: a feature commit updates its requirement.** The agent context
1629
(the authoring and verify skills and the always-on file `meta init` writes) now tells an
1730
agent that a commit adding, changing or removing behaviour updates the ledger in the same
@@ -32,6 +45,10 @@ here.**
3245
other cast. Schema comparison is TypeScript-owned, so no other port
3346
carries this comparison.
3447

48+
- **A report list in Java, Kotlin and Python no longer silently drops rows past the 50th.** A
49+
90-day daily report showed 50 days with no error when `limit` was omitted. C# also ignored a
50+
`limit` of 0 or below on a report instead of refusing it. See the report row cap under
51+
Changed.
3552
- **The api reference now follows a routes generator's `filter` as well as its `expose`.**
3653
`routesFile({ filter })` and `routesFileHono({ filter })` mount nothing for an entity the filter
3754
rejects, but the API pages still listed all five verbs for it. The pages now document the

‎docs/features/api-contract.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -206,15 +206,15 @@ feature demand (it is a consistency/safety divergence, not a capability).
206206
The TypeScript mount helpers (`@metaobjectsdev/runtime-ts/drizzle-fastify`,
207207
`/fastify` and `/hono`) pin five responses the contract leaves open — HTTP 5xx is
208208
implementation-defined below, and no corpus scenario sends a malformed body, a write the database rejects, a
209-
malformed page bound, or a value only the generated schema's format check refuses. All five use the contract's
209+
malformed page bound to an entity, or a value only the generated schema's format check refuses. All five use the contract's
210210
`{ "error": "<code>" }` envelope, and all five are scoped to the
211211
routes the helpers mount: an adopter's own routes, and a Fastify `setErrorHandler`
212212
or Hono `onError` the adopter installed, answer exactly as they did before.
213213

214214
| Response | When |
215215
|---|---|
216216
| malformed JSON body | a `POST`/`PATCH`/`PUT` body that does not parse as JSON (an empty body sent as `application/json` included) → HTTP 400 `{ "error": "invalid_json" }`. An EMPTY body on a method that takes none (`GET`, `DELETE`, `HEAD`) is accepted even when the request declares `content-type: application/json`, as many clients do on every request; a non-empty `DELETE` body is still parsed. Before, Fastify answered its own `{ "statusCode": 400, "code": "FST_ERR_CTP_INVALID_JSON_BODY", … }` and Hono a Zod `validation` error about a missing object |
217-
| malformed page bound | a `?limit=` or `?offset=` that is not a non-negative integer (`abc`, `-5`, `1.5`, an empty value) → HTTP 400 `{ "error": "pagination.invalid_value", "param": "limit", "value": "abc", "expected": "a non-negative integer (0 or more)" }`. Before, a non-numeric bound was dropped (every row came back) and a negative or fractional one reached SQL. There is no page-size maximum on an entity or a projection with declared columns; an opaque view (no declared columns, served by raw SQL) pages to at most 1000 rows, so there a `limit` above 1000 is refused too, with `"expected": "an integer from 0 to 1000"` (it used to be clamped silently). The same answer comes from the Drizzle mounts and the ObjectManager Fastify mount. No corpus scenario sends a malformed bound; the other ports answer it as their framework does |
217+
| malformed page bound | a `?limit=` or `?offset=` that is not a non-negative integer (`abc`, `-5`, `1.5`, an empty value) → HTTP 400 `{ "error": "pagination.invalid_value", "param": "limit", "value": "abc", "expected": "a non-negative integer (0 or more)" }`. Before, a non-numeric bound was dropped (every row came back) and a negative or fractional one reached SQL. There is no page-size maximum on an entity or a projection with declared columns; an opaque view (no declared columns, served by raw SQL) pages to at most 1000 rows, so there a `limit` above 1000 is refused too, with `"expected": "an integer from 0 to 1000"` (it used to be clamped silently). The same answer comes from the Drizzle mounts and the ObjectManager Fastify mount. The corpus sends a malformed bound only to a report (`report/bad-page-bound`), where every port answers this same envelope and also refuses `limit=0`; on an entity or a projection the other ports answer it as their framework does |
218218
| database constraint violation | a write the database itself rejects → `{ "error": "constraint_violation", "constraint": "<kind>" }`, HTTP **409** for `unique` and `foreign_key` (a conflict with another row) and HTTP **400** for `check` and `not_null` (the request's own value), per the status rule under "Error response". The body names the constraint KIND only — never the SQL, the table or the bound values. A `CHECK` the model derives (an enum, a `validator.numeric`/`length`/`regex` range) is normally refused earlier by the route's own validation, as the corpus's `{ "error": "validation" }` 400; this row is the database's backstop |
219219
| schema validation failure | a body that fails the insert/update schema → HTTP 400 `{ "error": "validation", "issues": [...] }`, each issue as Zod reports it **minus `pattern`**: Zod 4 attaches a failed `.regex()` check's source there, which for a generated `field.date` / `field.time` / `field.timestamp` check is a ~300-character calendar regex (and an authored `validator.regex` pattern is the server's to keep). The generated temporal checks carry a short `message` naming the expected format instead (`"must be an ISO date (YYYY-MM-DD)"`) |
220220
| unexpected server error | anything that is not a filter, validation, not-found or constraint answer — a query against a column the database no longer has, a driver failure → HTTP 500 `{ "error": "internal" }`, the code the cross-port reference servers already use. The body names no SQL, table, column or bound parameter; the full error goes to the server log (`console.error`). Before, Fastify's default handler echoed the driver message, which for Drizzle is the query text and its parameter values |
@@ -250,7 +250,13 @@ each handler they register instead; an `HTTPException` is rethrown to your
250250
- `limit=N` — page size.
251251
- `offset=N` — page offset. Both are non-negative integers; the TS mounts refuse
252252
anything else with a 400 (see "TS-only error responses").
253-
- **No page-size maximum, and the default is per port.** The contract pins what
253+
- **A report is the exception: one cap in every port.** A served `object.report` list returns every
254+
row when `limit` is omitted, up to 10,000 (a generator option), and answers
255+
`400 { "error": "report_too_large", "maxRows": N }` for one row more, or for a `limit` above the cap.
256+
A `limit` below 1, a negative `offset` or a value that is not an integer is
257+
`400 { "error": "pagination.invalid_value", "param": "limit" | "offset", "value", "expected" }` on
258+
a report in every port. See [reporting.md](reporting.md#how-a-report-is-served).
259+
- **No page-size maximum on an entity, and the default is per port.** The contract pins what
254260
an explicit `limit`/`offset` returns (`list-with-pagination`); it does not pin a
255261
default or a ceiling, and the ports differ. With `limit` **omitted**, the
256262
TypeScript mounts and the C# routes return **every** matching row, while the
@@ -528,7 +534,7 @@ the empty rows, `isNull=true` matches nothing, and a sort on it orders the same
528534
engine.
529535

530536
Gated by [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/)
531-
(27 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
537+
(30 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
532538
gives. The columns, their types and the per-port generated files are in
533539
[reporting.md](reporting.md#how-a-report-is-served).
534540

‎docs/features/reporting.md‎

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -556,8 +556,26 @@ rule every object uses ([api-contract.md](api-contract.md)).
556556
| a filter or sort error | The four field-naming envelopes of [api-contract.md](api-contract.md), unchanged. |
557557

558558
The route lists the compiled view. No request parameter picks dimensions, measures or a
559-
grain. The default page size is the port's own, as for any list route: TypeScript and C#
560-
return every row when `limit` is omitted, Java, Kotlin and Python the first 50.
559+
grain.
560+
561+
**Paging and the row cap.** A report is a whole answer, so its list route has no page default.
562+
With no `limit` it returns **every row**, in all five ports, up to one cap: **10,000 rows**. One
563+
row past the cap, or an explicit `limit` above it, is refused before any body is written with
564+
`400 { "error": "report_too_large", "maxRows": N }`; the route never truncates silently. Narrow
565+
the report with a filter, or page it with `limit` and `offset`. The cap is a generator option, the
566+
same default everywhere, set where each port sets its other generator options:
567+
568+
| Port | Set the cap |
569+
|---|---|
570+
| TypeScript | `routesFile({ reportMaxRows: 500 })` (and `routesFileHono`), or `reportMaxRows` on `mountReadOnlyCrudRoutes` in an owned routes file |
571+
| C# | `new RoutesGenerator { ReportMaxRows = 500 }` |
572+
| Java, Kotlin | the `reportMaxRows` generator arg (`<reportMaxRows>500</reportMaxRows>` in the plugin's `<args>`) |
573+
| Python | `router_generator(report_max_rows=500)` |
574+
575+
A page bound that is not usable is refused, never ignored: `limit` below 1, a negative `offset` or
576+
a value that is not an integer answers `400 { "error": "pagination.invalid_value", "param": ... }`,
577+
the envelope the TypeScript mounts use for a malformed bound. Entity and projection lists keep
578+
their own paging ([api-contract.md](api-contract.md#sort--pagination)).
561579

562580
**What a caller may filter and sort on.** Every derived field whose type has filter operators
563581
(string, enum, uuid, int, long, double, float, decimal, currency, date, time, timestamp,
@@ -716,7 +734,7 @@ compares the canonical string. D1 has no separate corpus: it is SQLite behind Cl
716734
binding, so TypeScript runs the same scenarios and the same REST scenarios through Miniflare's
717735
local D1 (`meta migrate --dialect d1`), with no cloud account.
718736

719-
The REST surface is gated by twenty-seven scenarios under
737+
The REST surface is gated by thirty scenarios under
720738
[`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/),
721739
run in the **generated lane on all five ports**: list (a dimension with a segment-scoped sum
722740
that is null for one group), a time dimension at a grain (`YYYY-MM-DD`), a no-dimension
@@ -727,7 +745,9 @@ sale keeps its row, with a filter and a sort on its defaulted measure, a dimensi
727745
two references, one time dimension listed at two grains, a filter on a string (`like`), a boolean and a
728746
uuid dimension, a filter on a measure that carries its own `@filter`, and a dimension over an
729747
array and one over a map, which refuse a filter (`invalid_filter_field`) and a sort
730-
(`invalid_sort`). The corpus model carries one sourceless report, so a port that serves every
748+
(`invalid_sort`), and the row cap: a report of 61 rows served with a cap of 60 returns 60 rows past
749+
the old 50-row default, refuses the whole report, an explicit limit above the cap and a bad page
750+
bound. The corpus model carries one sourceless report, so a port that serves every
731751
report it finds fails. Every port runs the scenarios through its generated routes against the
732752
real views on Postgres, SQLite and MySQL (`report/schema.sqlite.sql`, `report/schema.mysql.sql`),
733753
and TypeScript also through local D1. A report's decimal columns are asserted as canonical

‎examples/advanced-modeling/codegen/generators/routes.ts‎

Lines changed: 11 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,6 +107,8 @@ function renderRoutes(
107107
// which emits no `expose` key at all and so keeps output byte-identical for projects
108108
// that do not use it.
109109
expose?: readonly CrudVerb[],
110+
// Row cap of a served report (`routesFile({ reportMaxRows })`). undefined emits no key.
111+
reportMaxRows?: number,
110112
): string {
111113
// FR-017 Tier 2 — a TPH discriminator base mounts polymorphic list/get at the
112114
// base path plus a full per-subtype CRUD route set scoped to each
@@ -160,6 +162,7 @@ function renderRoutes(
160162
: "") +
161163
(keyless ? `\n${indent}itemRoutes: false,` : "") +
162164
(report ? `\n${indent}resource: "report",` : "") +
165+
(report && reportMaxRows !== undefined ? `\n${indent}reportMaxRows: ${reportMaxRows},` : "") +
163166
decimalColumnsLine(entity, indent, ctx.loadedRoot);
164167
const FastifyInstanceSym = imp("t:FastifyInstance@fastify");
165168
const mountReadOnlyCrudRoutesSym = imp(`mountReadOnlyCrudRoutes@${runtimeSpec}`);
@@ -705,6 +708,13 @@ export interface RoutesFileOpts {
705708
* your host file. Off by default, so a project that does not ask gets no new file.
706709
*/
707710
registerAll?: boolean;
711+
/**
712+
* The row cap of a report list route (default 10000, the same in every port). With no
713+
* `limit` a report answers every row up to the cap and is refused above it with
714+
* `400 { "error": "report_too_large", "maxRows": N }`; an explicit `limit` above it is
715+
* refused the same way. Absent emits no key, so the output stays byte-identical.
716+
*/
717+
reportMaxRows?: number;
708718
/**
709719
* Where the emitted routes import their mount helpers from. Absent: the adapter copy
710720
* `meta eject routes` placed in `codegen/runtime/`, by a path computed from the target's
@@ -728,7 +738,7 @@ export const routesFile = function routesFile(opts?: RoutesFileOpts): Generator
728738
};
729739
return {
730740
path: entityOutputPath(ctx.config.outputLayout ?? "flat", effectivePackage(entity), `${entity.name}.routes.ts`),
731-
content: await formatTs(renderRoutes(entity, renderContext, resolveExpose(entity, opts?.expose))),
741+
content: await formatTs(renderRoutes(entity, renderContext, resolveExpose(entity, opts?.expose), opts?.reportMaxRows)),
732742
};
733743
});
734744
const generator: Generator = {

‎examples/advanced-modeling/codegen/runtime/drizzle-fastify/filter-parser.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import { FilterParseError, parsePageBound, SORT_EXPECTED } from "./list-params.j
1010

1111
// Re-exported so existing importers keep one door; the definitions live in the
1212
// drizzle-free list-params module, which the ObjectManager Fastify mount also uses.
13-
export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, SORT_EXPECTED } from "./list-params.js";
13+
export { FilterParseError, parsePageBound, PAGINATION_EXPECTED, RAW_VIEW_MAX_LIMIT, REPORT_MAX_ROWS, ReportTooLargeError, parseReportPage, SORT_EXPECTED } from "./list-params.js";
1414
import {
1515
ANY_TEMPORAL_EXPECTED, FORMAT_EXPECTED, PLUS_OFFSET_HINT,
1616
looksLikeUnencodedPlusOffset, matchesAnyTemporal, matchesFormat, utcIsoIfZoned, utcIsoInstant,

‎examples/advanced-modeling/codegen/runtime/drizzle-fastify/list-params.ts‎

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,11 @@ export class FilterParseError extends Error {
1515
* no allowlist to page through and returns at most this many rows. */
1616
export const RAW_VIEW_MAX_LIMIT = 1000;
1717

18+
/** The most rows a report list answers, whether or not the caller sends a `limit`. A report
19+
* with no `limit` returns every row up to this cap and is refused above it (never
20+
* truncated); the same default in all five ports. */
21+
export const REPORT_MAX_ROWS = 10_000;
22+
1823
/** The `expected` member of a `pagination.invalid_value` answer with no upper bound. */
1924
export const PAGINATION_EXPECTED = "a non-negative integer (0 or more)";
2025

@@ -48,3 +53,47 @@ export function parsePageBound(
4853

4954
/** The `expected` member of an `invalid_sort` answer — the one spelling `?sort` takes. */
5055
export const SORT_EXPECTED = "sort=<field>:asc|desc";
56+
57+
/** The page a report list reads: `limit` is how many rows to FETCH, which is one past the
58+
* cap when the caller sent none, so the caller can tell exactly the cap apart from more than it. */
59+
export interface ReportPage {
60+
readonly limit: number;
61+
readonly offset: number;
62+
/** True when the caller sent no `limit`, so more than `maxRows` rows means refuse. */
63+
readonly unbounded: boolean;
64+
}
65+
66+
/** A report answered more rows than `maxRows` allows. The wire answer is
67+
* `400 { "error": "report_too_large", "maxRows": N }`. */
68+
export class ReportTooLargeError extends Error {
69+
constructor(public readonly maxRows: number) {
70+
super(`A report answers at most ${maxRows} rows; narrow it with a filter or page it with ?limit=.`);
71+
this.name = "ReportTooLargeError";
72+
}
73+
}
74+
75+
/**
76+
* Read a report list's `?limit=` / `?offset=`. A limit is an integer of at least 1 (a
77+
* report with `limit=0` is a bad bound, not an empty answer) and an offset is a
78+
* non-negative integer; anything else is `pagination.invalid_value`. A limit above
79+
* `maxRows` throws {@link ReportTooLargeError}.
80+
*/
81+
export function parseReportPage(query: Record<string, unknown>, maxRows: number): ReportPage {
82+
const badLimit = (value: string) =>
83+
new FilterParseError(
84+
"pagination.invalid_value",
85+
`?limit= must be an integer from 1 to ${maxRows}, got "${value}".`,
86+
{ param: "limit", value, expected: `an integer from 1 to ${maxRows}` },
87+
);
88+
let limit: number | undefined;
89+
try {
90+
limit = parsePageBound(query, "limit");
91+
} catch (e) {
92+
if (e instanceof FilterParseError) throw badLimit(String(query.limit));
93+
throw e;
94+
}
95+
if (limit === 0) throw badLimit("0");
96+
if (limit !== undefined && limit > maxRows) throw new ReportTooLargeError(maxRows);
97+
const offset = parsePageBound(query, "offset") ?? 0;
98+
return { limit: limit ?? maxRows + 1, offset, unbounded: limit === undefined };
99+
}

0 commit comments

Comments
 (0)