Skip to content

Commit de4f9f6

Browse files
authored
feat: add CSV, JSON and XML export to report list route (#471)
* feat(reporting): export a report as CSV, JSON or XML via ?format= in all five ports Adds the standalone report-export catalog generator (importable and ejectable in every port), the ?format= branch on report list routes, the invalid_format, export_too_large and export_with_count envelopes, a maxRows cap that never truncates, a CSV formula-injection guard, and six shared api-contract scenarios. ADR-0031 Amendment 1 records that JSON and XML come from each port's standard serializer for now. Excel (xlsx) is reserved by name only. * docs: catalog note for the report-export generator * fix: single Unreleased heading; set export headers only after the body renders * refactor: drop dead export helpers * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Narrow export withCount refusal, wire-aware JVM export cells * no-mistakes(document): Doc pass for report export. Fixed stale corpus and generator counts. Export contract, ports, ADR-0031 Amendment 1, roadmap and changelog already match code at target. Edited four files: docs/CONFORMANCE.md, docs/features/api-contract.md, docs/ports/kotlin.md, README.md. No code or tests touched.
1 parent 1878943 commit de4f9f6

135 files changed

Lines changed: 6798 additions & 200 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎CHANGELOG.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,29 @@ here.**
1010

1111
## [Unreleased]
1212

13+
### Added
14+
15+
- **Report export: `GET /<segment>?format=csv|json|xml` on a served report, in all five ports.**
16+
A new standalone `report-export` catalog generator (importable as shipped, ejectable in every
17+
port, drift-checked by `verify --codegen`, never implied by a library) generates one export
18+
module per served report plus one shared writer module per project, and makes the routes
19+
generator emit the `format` branch. Without `format`, or with no export generator wired, a
20+
response is byte-identical to before. CSV is RFC 4180 with CRLF, a header row of the derived
21+
field names, wire-form values and a formula-injection guard on every string cell (the render
22+
escapers' rule); JSON is the list route's body (the array, or the `withCount` envelope) plus
23+
`Content-Disposition`; XML is `<{Report}><row><{field}>...` with a pinnable null form
24+
(`xmlNulls`). A format the route does not wire answers
25+
`400 {"error":"invalid_format","expected":[...]}`, `withCount` with `format=csv` or `format=xml`
26+
answers `400 export_with_count`, and more rows than the generator option `maxRows` (default 100,000)
27+
answers `400 {"error":"export_too_large","maxRows":N}` before a byte is written; nothing is ever
28+
truncated. With `format` set `maxRows` governs and the list route's `report_too_large` cap does
29+
not. The export adds no dependency an adopter must install in any port, and `xlsx` is reserved
30+
(refused today like any unwired format). Pinned by six `report/scenarios/export-*.yaml`
31+
scenarios run by every port's generated report lane. See
32+
[docs/features/reporting.md](docs/features/reporting.md#exporting-a-report),
33+
[docs/features/api-contract.md](docs/features/api-contract.md#report-export) and
34+
[ADR-0031 Amendment 1](spec/decisions/ADR-0031-serialization-vs-data-download-boundary.md).
35+
1336
## [1.1.4] — 2026-10-26
1437

1538
_npm `1.1.4` (full lockstep across all 14 `@metaobjectsdev/*` publish candidates), PyPI `1.1.4`, NuGet `1.1.4` and Maven Central `8.1.4`._

‎README.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@ first-week wedge plan — and `meta init` picks up from there.
148148
|---|---|---|---|
149149
| TypeScript | **npm `1.1.4`** — the `@metaobjectsdev/*` packages | [`docs/ports/typescript.md`](docs/ports/typescript.md) | [`server/typescript/`](server/typescript/) · [`client/web/`](client/web/) |
150150
| Java | **Maven Central `8.1.4`** (`com.metaobjects:*` — the JVM major is npm major + 7) — loader + OMDB + render + Maven plugin all shipped; full conformance green | [`docs/ports/java.md`](docs/ports/java.md) | [`server/java/`](server/java/) |
151-
| Kotlin | **Maven Central `8.1.4`** — codegen tier on top of Java: 16 generators (entity, Exposed table, names, relations, repository, output-parser, output-prompt, render-helper, extractor, filter-allowlist, validator, Spring config, storedProc, Spring controller, requirement-tests, api-docs); 24 / 24 persistence-conformance | [`docs/ports/kotlin.md`](docs/ports/kotlin.md) | [`server/java/codegen-kotlin/`](server/java/codegen-kotlin/) · [`server/java/metadata-ktx/`](server/java/metadata-ktx/) |
151+
| Kotlin | **Maven Central `8.1.4`** — codegen tier on top of Java: 17 generators (entity, Exposed table, names, relations, repository, output-parser, output-prompt, render-helper, extractor, filter-allowlist, validator, Spring config, storedProc, Spring controller, requirement-tests, api-docs, report-export); 24 / 24 persistence-conformance | [`docs/ports/kotlin.md`](docs/ports/kotlin.md) | [`server/java/codegen-kotlin/`](server/java/codegen-kotlin/) · [`server/java/metadata-ktx/`](server/java/metadata-ktx/) |
152152
| C# | **NuGet `1.1.4`** — loader + conformance + EF Core codegen + render engine + `dotnet meta` CLI all shipped | [`docs/ports/csharp.md`](docs/ports/csharp.md) | [`server/csharp/`](server/csharp/) |
153153
| Python | **PyPI `1.1.4`** — loader + conformance + render + entity-model codegen + ObjectManager runtime shipped; schema migrations are TS-owned (ADR-0015) | [`docs/ports/python.md`](docs/ports/python.md) | [`server/python/`](server/python/) |
154154

‎docs/CONFORMANCE.md‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
3434
| [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ |
3535
| [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ |
3636
| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) |
37-
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 106 (31 core + 10 tph + 9 m2m + 4 jsonb + 2 write-through + 12 projection + 27 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
37+
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 117 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
3838
| [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) |
3939
| [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) |
4040
| [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ |
@@ -348,9 +348,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc
348348
`report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the
349349
write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates).
350350

351-
### `fixtures/api-contract-conformance/` (106)
351+
### `fixtures/api-contract-conformance/` (117)
352352

353-
All 106 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
353+
All 117 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
354354
REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD
355355
routes answer identically over HTTP — list / get / create / patch+put / delete,
356356
plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1`
@@ -364,12 +364,12 @@ URL grammar `?filter[<field>][<op>]=<value>` (FR-009). On top of the 31 core
364364
scenarios the corpus carries seven sub-corpora — `tph/` (10, single-table
365365
inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination
366366
each corpus alone could not reach, and 1 pinning the collection-URL spelling),
367-
`jsonb/` (4, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
367+
`jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
368368
`write-through/` (2, table-write + view-read entities), `projection/` (12, a
369369
read-only view answers reads and refuses writes with 405; a projection with no declared
370370
identity has no item route; a projection keyed on a field not named `id` is addressed by
371371
it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one
372-
canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (27,
372+
canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (37,
373373
FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on
374374
its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps
375375
a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java,
@@ -555,7 +555,7 @@ files into a real Cube.
555555
## Orphaned fixtures (tested but not yet documented)
556556

557557
The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37
558-
+ render 16 + persistence 59 + api-contract 106 + source-resolution 25 + scope 10 +
558+
+ render 16 + persistence 59 + api-contract 117 + source-resolution 25 + scope 10 +
559559
dependency 23) each map to a feature doc, and so do the two requirement corpora and the
560560
`cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining
561561
corpora in the totals table gate tooling contracts (registry manifests, provider

‎docs/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ this tree is documentation, not the source of truth.
7272
| Adopt a design MetaObjects already ships — users/groups/roles, an LLM trace envelope — instead of authoring it (`libraries`, `meta eject <library>`) | [`features/libraries.md`](features/libraries.md) |
7373
| Record what the system is supposed to do, and stop agents reviving retired features | [`features/requirements.md`](features/requirements.md) |
7474
| Declare what a dashboard groups by and counts (`dimension`, `measure`, `segment`, `object.report`; load-time checked; a view-backed report becomes a SQL view served by a generated read-only list route and a TypeScript list hook) | [`features/reporting.md`](features/reporting.md) |
75+
| Download a report as CSV, JSON or XML (`?format=`, the `report-export` generator, its cap and null forms) | [`features/reporting.md`](features/reporting.md#exporting-a-report) |
7576
| Export the reporting vocabulary to Cube (`cube-model`: cubes, joins, rollups; the `cube` lane that checks it against a real Cube) | [`features/cube-export.md`](features/cube-export.md) |
7677
| Wire prompt construction (FR-004) | [`features/templates-and-payloads.md`](features/templates-and-payloads.md) |
7778
| Share a metadata shape across multiple instances (abstracts, `extends:`) | [`features/abstracts-and-inheritance.md`](features/abstracts-and-inheritance.md) |

‎docs/features/api-contract.md‎

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,9 @@ each handler they register instead; an `HTTPException` is rethrown to your
266266
A `limit` below 1, a negative `offset` or a value that is not an integer is
267267
`400 { "error": "pagination.invalid_value", "param": "limit" | "offset", "value", "expected" }` on
268268
a report in every port. See [reporting.md](reporting.md#how-a-report-is-served).
269+
- **A report list can be exported: `?format=csv|json|xml`.** See
270+
[Report export](#report-export) below. Without `format` the response is byte-identical to the list
271+
route described here.
269272
- **No page-size maximum on an entity, and the default is per port.** The contract pins what
270273
an explicit `limit`/`offset` returns (`list-with-pagination`); it does not pin a
271274
default or a ceiling, and the ports differ. With `limit` **omitted**, the
@@ -281,6 +284,63 @@ each handler they register instead; an `HTTPException` is rethrown to your
281284
`[<row>...]` to `{ rows: [<row>...], total: <N> }` (needed for grid
282285
pagination). The grid hook always sends `withCount=1`.
283286

287+
### Report export
288+
289+
One selector on the existing report list route: `GET <apiPrefix>/<segment>?format=csv|json|xml`.
290+
The route accepts `format` only when a `report-export` generator is wired for the project (see
291+
[reporting.md](reporting.md#exporting-a-report)); a project that wires none sees no change at all,
292+
and `?format=` is then ignored exactly as any unknown query parameter is today. **Accept headers are
293+
never consulted**: the selector is the query parameter, in every port.
294+
295+
| Request | Answer |
296+
|---|---|
297+
| no `format` | Exactly the list route above: same body, same headers, same caps. |
298+
| `format=json` | The list route's JSON body, unchanged, plus `Content-Disposition`: the array, or with `withCount=1` the `{ rows, total }` envelope. `Content-Type: application/json`. |
299+
| `format=csv` | RFC 4180 CSV. `Content-Type: text/csv; charset=utf-8`. |
300+
| `format=xml` | `application/xml`, shape below. |
301+
| any other value (`xlsx`, `pdf`, `CSV`, empty) | `400 { "error": "invalid_format", "expected": ["csv", "json", "xml"] }`. `expected` lists the formats this route wires, in that order. `xlsx` is reserved for a later generator (`report-export-xlsx`) and is refused today like any unknown value. Matching is case-sensitive. |
302+
| `format=csv` or `format=xml` and `withCount` truthy | `400 { "error": "export_with_count" }` (`message` is free prose). Only JSON carries the `{ rows, total }` envelope. |
303+
| more rows than `maxRows` | `400 { "error": "export_too_large", "maxRows": N }` before a byte of the body is written. |
304+
305+
- **Header comparison.** Header names are case-insensitive. A runner compares `Content-Type` with the whitespace around `;` normalized (`text/csv;charset=utf-8` equals `text/csv; charset=utf-8`), because some servers re-spell it; every other header is compared exactly.
306+
- **Headers.** Every successful export carries
307+
`Content-Disposition: attachment; filename="<segment>.<ext>"` with `<ext>` one of `csv`, `json`,
308+
`xml`. A response without `format` carries none.
309+
- **Filters, sort, `limit` and `offset`** use the list route's parser, allowlists and 400 envelopes
310+
unchanged. In export mode an absent `limit` means every row up to `maxRows`; an explicit `limit` or
311+
`offset` is honoured.
312+
- **One cap, two names.** `maxRows` is an option of the `report-export` generator, default 100,000.
313+
The route fetches `maxRows + 1` rows and refuses with `export_too_large` rather than truncate, and
314+
an explicit `limit` above `maxRows` is refused the same way. With `?format` set the export cap
315+
governs and `report_too_large` never answers; without it the list route's own cap
316+
(`report_too_large`, default 10,000) is untouched. A report is materialised up to the cap; there is
317+
no streaming in this version.
318+
- **CSV.** UTF-8, no BOM unless the generated constant `EXPORT_CSV_BOM` is flipped to true, CRLF after
319+
every record including the last. One header row with the report's derived field names, in derived-field
320+
order (dimensions, then measures, as `reportShape` lists them). A value is written in its wire
321+
form: currency as integer minor units, a date as `YYYY-MM-DD`, a decimal as the `wire` setting spells
322+
it, a boolean as `true`/`false`, null as an empty cell. A cell containing `"`, `,`, CR or LF is
323+
quoted with embedded quotes doubled. **The formula-injection guard applies to every string-valued
324+
cell** (`field.string`, an enum symbol, a uri): a value starting with `=`, `+`, `-`, `@`, TAB or CR is
325+
prefixed with `'` before quoting. This is the render escapers' rule
326+
(`render-csv-injection` fixture). Numbers, decimals, dates and booleans are never guarded, so a
327+
negative number stays a number.
328+
- **XML.** `<{ReportName}><row><{field}>value</{field}>...</row>...</{ReportName}>`, no XML declaration,
329+
no indentation, UTF-8, values in the same wire form as CSV, text and attribute content escaped
330+
(`&amp; &lt; &gt;`). There is no injection guard in XML. A null follows the generator option
331+
`xmlNulls`: `"nil"` writes the field as an empty element with `xsi:nil="true"` and declares
332+
`xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"` on the root; `"empty"` writes an empty element
333+
and declares nothing. **The default is each port's own standard** (ADR-0019 Amendment 2): see the
334+
per-port table in [reporting.md](reporting.md#exporting-a-report). The corpus pins `"nil"` in every lane
335+
and compares the body as an XML infoset (names, attributes, text, order), not byte for byte.
336+
- **JSON** is whatever the list route's standard serializer writes in that port, so a `wire` setting
337+
applies to it exactly as it does to the list route.
338+
339+
The report-export scenarios (`report/scenarios/export-*.yaml`) use three assertion keys the rest of the
340+
corpus does not: `headers` on `expect` (a map of lower-cased header name to its exact value, or `null` for
341+
"the header is absent"), `text` on the body (the exact body, compared as UTF-8 text), and `xml` on the body
342+
(compared as an XML infoset). A request may also carry `headers` (a map sent with the request).
343+
284344
### Base-URL policy
285345

286346
The two sides are configured **separately, and deliberately so**.
@@ -544,7 +604,7 @@ the empty rows, `isNull=true` matches nothing, and a sort on it orders the same
544604
engine.
545605

546606
Gated by [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/)
547-
(30 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
607+
(37 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
548608
gives. The columns, their types and the per-port generated files are in
549609
[reporting.md](reporting.md#how-a-report-is-served).
550610

‎docs/features/own-your-codegen.md‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1143,6 +1143,17 @@ from which the goal infers it. `requirement-tests` is one such name:
11431143
`KotlinRequirementTestsGenerator.kt`, each with its default rendering. The identity
11441144
function, the digest and the renderer and filter hook types stay in the package.
11451145

1146+
### `report-export` (all ports)
1147+
1148+
`report-export` is a standalone catalog generator, never implied by a library, that adds the
1149+
`?format=csv|json|xml` download to served report list routes. Every port ships it and ejects it
1150+
with its own command (`meta eject report-export`, `metaobjects eject`, `mvn metaobjects:eject
1151+
-Dnames=report-export`, `dotnet meta eject report-export`). The copy holds the CSV and XML writers,
1152+
so the guard, quoting and null form are yours to change. The routes generator emits the `format`
1153+
branch only when the generator is wired: every port's runner detects it from the generator list (on
1154+
Java and Kotlin the Maven plugin hands that list to `routes`). Options and defaults:
1155+
[reporting.md](reporting.md#exporting-a-report).
1156+
11461157
### Java `entity` and `extractor`
11471158

11481159
```bash

0 commit comments

Comments
 (0)