You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit de4f9f6
Browse filesBrowse the repository at this point in the historyBrowse files
feat: add CSV, JSON and XML export to report list route (#471)
* feat(reporting): export a report as CSV, JSON or XML via ?format= in all five ports
Adds the standalone report-export catalog generator (importable and ejectable in
every port), the ?format= branch on report list routes, the invalid_format,
export_too_large and export_with_count envelopes, a maxRows cap that never
truncates, a CSV formula-injection guard, and six shared api-contract scenarios.
ADR-0031 Amendment 1 records that JSON and XML come from each port's standard
serializer for now. Excel (xlsx) is reserved by name only.
* docs: catalog note for the report-export generator
* fix: single Unreleased heading; set export headers only after the body renders
* refactor: drop dead export helpers
* no-mistakes(review): apply reviewer inline fixes
* no-mistakes(review): Narrow export withCount refusal, wire-aware JVM export cells
* no-mistakes(document): Doc pass for report export. Fixed stale corpus and generator counts. Export contract, ports, ADR-0031 Amendment 1, roadmap and changelog already match code at target. Edited four files: docs/CONFORMANCE.md, docs/features/api-contract.md, docs/ports/kotlin.md, README.md. No code or tests touched.
| Java |**Maven Central `8.1.4`** (`com.metaobjects:*` — the JVM major is npm major + 7) — loader + OMDB + render + Maven plugin all shipped; full conformance green |[`docs/ports/java.md`](docs/ports/java.md)|[`server/java/`](server/java/)|
151
-
| Kotlin |**Maven Central `8.1.4`** — codegen tier on top of Java: 16 generators (entity, Exposed table, names, relations, repository, output-parser, output-prompt, render-helper, extractor, filter-allowlist, validator, Spring config, storedProc, Spring controller, requirement-tests, api-docs); 24 / 24 persistence-conformance |[`docs/ports/kotlin.md`](docs/ports/kotlin.md)|[`server/java/codegen-kotlin/`](server/java/codegen-kotlin/) · [`server/java/metadata-ktx/`](server/java/metadata-ktx/)|
151
+
| Kotlin |**Maven Central `8.1.4`** — codegen tier on top of Java: 17 generators (entity, Exposed table, names, relations, repository, output-parser, output-prompt, render-helper, extractor, filter-allowlist, validator, Spring config, storedProc, Spring controller, requirement-tests, api-docs, report-export); 24 / 24 persistence-conformance |[`docs/ports/kotlin.md`](docs/ports/kotlin.md)|[`server/java/codegen-kotlin/`](server/java/codegen-kotlin/) · [`server/java/metadata-ktx/`](server/java/metadata-ktx/)|
|[`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/)| 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) |
37
-
|[`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/)|106 (31 core + 10 tph + 9 m2m + 4 jsonb + 2 write-through + 12 projection + 27 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
37
+
|[`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/)|117 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
@@ -348,9 +348,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc
348
348
`report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the
349
349
write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates).
350
350
351
-
### `fixtures/api-contract-conformance/` (106)
351
+
### `fixtures/api-contract-conformance/` (117)
352
352
353
-
All 106 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
353
+
All 117 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
354
354
REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD
355
355
routes answer identically over HTTP — list / get / create / patch+put / delete,
356
356
plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1`
@@ -364,12 +364,12 @@ URL grammar `?filter[<field>][<op>]=<value>` (FR-009). On top of the 31 core
364
364
scenarios the corpus carries seven sub-corpora — `tph/` (10, single-table
365
365
inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination
366
366
each corpus alone could not reach, and 1 pinning the collection-URL spelling),
367
-
`jsonb/` (4, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
367
+
`jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
368
368
`write-through/` (2, table-write + view-read entities), `projection/` (12, a
369
369
read-only view answers reads and refuses writes with 405; a projection with no declared
370
370
identity has no item route; a projection keyed on a field not named `id` is addressed by
371
371
it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one
372
-
canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (27,
372
+
canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (37,
373
373
FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on
374
374
its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps
375
375
a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java,
@@ -555,7 +555,7 @@ files into a real Cube.
555
555
## Orphaned fixtures (tested but not yet documented)
556
556
557
557
The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37
Copy file name to clipboardExpand all lines: docs/README.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -72,6 +72,7 @@ this tree is documentation, not the source of truth.
72
72
| Adopt a design MetaObjects already ships — users/groups/roles, an LLM trace envelope — instead of authoring it (`libraries`, `meta eject <library>`) |[`features/libraries.md`](features/libraries.md)|
73
73
| Record what the system is supposed to do, and stop agents reviving retired features |[`features/requirements.md`](features/requirements.md)|
74
74
| Declare what a dashboard groups by and counts (`dimension`, `measure`, `segment`, `object.report`; load-time checked; a view-backed report becomes a SQL view served by a generated read-only list route and a TypeScript list hook) |[`features/reporting.md`](features/reporting.md)|
75
+
| Download a report as CSV, JSON or XML (`?format=`, the `report-export` generator, its cap and null forms) |[`features/reporting.md`](features/reporting.md#exporting-a-report)|
75
76
| Export the reporting vocabulary to Cube (`cube-model`: cubes, joins, rollups; the `cube` lane that checks it against a real Cube) |[`features/cube-export.md`](features/cube-export.md)|
76
77
| Wire prompt construction (FR-004) |[`features/templates-and-payloads.md`](features/templates-and-payloads.md)|
77
78
| Share a metadata shape across multiple instances (abstracts, `extends:`) |[`features/abstracts-and-inheritance.md`](features/abstracts-and-inheritance.md)|
a report in every port. See [reporting.md](reporting.md#how-a-report-is-served).
269
+
-**A report list can be exported: `?format=csv|json|xml`.** See
270
+
[Report export](#report-export) below. Without `format` the response is byte-identical to the list
271
+
route described here.
269
272
-**No page-size maximum on an entity, and the default is per port.** The contract pins what
270
273
an explicit `limit`/`offset` returns (`list-with-pagination`); it does not pin a
271
274
default or a ceiling, and the ports differ. With `limit`**omitted**, the
@@ -281,6 +284,63 @@ each handler they register instead; an `HTTPException` is rethrown to your
281
284
`[<row>...]` to `{ rows: [<row>...], total: <N> }` (needed for grid
282
285
pagination). The grid hook always sends `withCount=1`.
283
286
287
+
### Report export
288
+
289
+
One selector on the existing report list route: `GET <apiPrefix>/<segment>?format=csv|json|xml`.
290
+
The route accepts `format` only when a `report-export` generator is wired for the project (see
291
+
[reporting.md](reporting.md#exporting-a-report)); a project that wires none sees no change at all,
292
+
and `?format=` is then ignored exactly as any unknown query parameter is today. **Accept headers are
293
+
never consulted**: the selector is the query parameter, in every port.
294
+
295
+
| Request | Answer |
296
+
|---|---|
297
+
| no `format`| Exactly the list route above: same body, same headers, same caps. |
298
+
|`format=json`| The list route's JSON body, unchanged, plus `Content-Disposition`: the array, or with `withCount=1` the `{ rows, total }` envelope. `Content-Type: application/json`. |
| any other value (`xlsx`, `pdf`, `CSV`, empty) |`400 { "error": "invalid_format", "expected": ["csv", "json", "xml"] }`. `expected` lists the formats this route wires, in that order. `xlsx` is reserved for a later generator (`report-export-xlsx`) and is refused today like any unknown value. Matching is case-sensitive. |
302
+
|`format=csv` or `format=xml` and `withCount` truthy |`400 { "error": "export_with_count" }` (`message` is free prose). Only JSON carries the `{ rows, total }` envelope. |
303
+
| more rows than `maxRows`|`400 { "error": "export_too_large", "maxRows": N }` before a byte of the body is written. |
304
+
305
+
-**Header comparison.** Header names are case-insensitive. A runner compares `Content-Type` with the whitespace around `;` normalized (`text/csv;charset=utf-8` equals `text/csv; charset=utf-8`), because some servers re-spell it; every other header is compared exactly.
306
+
-**Headers.** Every successful export carries
307
+
`Content-Disposition: attachment; filename="<segment>.<ext>"` with `<ext>` one of `csv`, `json`,
308
+
`xml`. A response without `format` carries none.
309
+
-**Filters, sort, `limit` and `offset`** use the list route's parser, allowlists and 400 envelopes
310
+
unchanged. In export mode an absent `limit` means every row up to `maxRows`; an explicit `limit` or
311
+
`offset` is honoured.
312
+
-**One cap, two names.**`maxRows` is an option of the `report-export` generator, default 100,000.
313
+
The route fetches `maxRows + 1` rows and refuses with `export_too_large` rather than truncate, and
314
+
an explicit `limit` above `maxRows` is refused the same way. With `?format` set the export cap
315
+
governs and `report_too_large` never answers; without it the list route's own cap
316
+
(`report_too_large`, default 10,000) is untouched. A report is materialised up to the cap; there is
317
+
no streaming in this version.
318
+
-**CSV.** UTF-8, no BOM unless the generated constant `EXPORT_CSV_BOM` is flipped to true, CRLF after
319
+
every record including the last. One header row with the report's derived field names, in derived-field
320
+
order (dimensions, then measures, as `reportShape` lists them). A value is written in its wire
321
+
form: currency as integer minor units, a date as `YYYY-MM-DD`, a decimal as the `wire` setting spells
322
+
it, a boolean as `true`/`false`, null as an empty cell. A cell containing `"`, `,`, CR or LF is
323
+
quoted with embedded quotes doubled. **The formula-injection guard applies to every string-valued
324
+
cell** (`field.string`, an enum symbol, a uri): a value starting with `=`, `+`, `-`, `@`, TAB or CR is
325
+
prefixed with `'` before quoting. This is the render escapers' rule
326
+
(`render-csv-injection` fixture). Numbers, decimals, dates and booleans are never guarded, so a
327
+
negative number stays a number.
328
+
-**XML.**`<{ReportName}><row><{field}>value</{field}>...</row>...</{ReportName}>`, no XML declaration,
329
+
no indentation, UTF-8, values in the same wire form as CSV, text and attribute content escaped
330
+
(`& < >`). There is no injection guard in XML. A null follows the generator option
331
+
`xmlNulls`: `"nil"` writes the field as an empty element with `xsi:nil="true"` and declares
332
+
`xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"` on the root; `"empty"` writes an empty element
333
+
and declares nothing. **The default is each port's own standard** (ADR-0019 Amendment 2): see the
334
+
per-port table in [reporting.md](reporting.md#exporting-a-report). The corpus pins `"nil"` in every lane
335
+
and compares the body as an XML infoset (names, attributes, text, order), not byte for byte.
336
+
-**JSON** is whatever the list route's standard serializer writes in that port, so a `wire` setting
337
+
applies to it exactly as it does to the list route.
338
+
339
+
The report-export scenarios (`report/scenarios/export-*.yaml`) use three assertion keys the rest of the
340
+
corpus does not: `headers` on `expect` (a map of lower-cased header name to its exact value, or `null` for
341
+
"the header is absent"), `text` on the body (the exact body, compared as UTF-8 text), and `xml` on the body
342
+
(compared as an XML infoset). A request may also carry `headers` (a map sent with the request).
343
+
284
344
### Base-URL policy
285
345
286
346
The two sides are configured **separately, and deliberately so**.
@@ -544,7 +604,7 @@ the empty rows, `isNull=true` matches nothing, and a sort on it orders the same
544
604
engine.
545
605
546
606
Gated by [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/)
547
-
(30 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
607
+
(37 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
548
608
gives. The columns, their types and the per-port generated files are in
0 commit comments