Skip to content

Commit b36adc7

Browse files
authored
fix: replace regex and string hacks with real parsers across ports (#511)
* fix: replace regex and string-rewrite matching with authoritative parsers across ports (audit findings 5, 7-15, 17, 18) Currency parsing via Intl, SQL splitter, view comparators, Python YAML 1.2, Mustache tag scanning, forgiving XML reader, JSONC-aware upgrade rewriter, leak-scan renames, Java authored-name serialization, ERR_COLUMN_NAME_COLLISION, pnpm workspace parsing and assorted smaller instances. * fix: import the shared support class in the Spring and Kotlin filter-allowlist generators An ejected copy is package-renamed, so same-package bare-name resolution stopped compiling once #503 started calling the support class there. * fix: Roslyn namespace/handoff detection, ASCII Pydantic patterns, route-segment collision (audit finding 18 follow-ups) * feat: enum strip fold-collision error, numeric CSV exemption, one filter glob grammar in Java ERR_ENUM_NORMALIZE_COLLISION in all ports; CSV formula guard skips strict numeric literals; <filters> use Scope.compilePattern semantics and fail generation when a pattern would select fewer objects than before. * fix(render,cli): decode every JSON escape and close fences properly in extract; make eject --undo order-sensitive The forgiving JSON reader dropped \uXXXX, \b and \f escapes and the fence matcher closed at the first inner triple backtick, in all five ports. eject --undo treated a copy whose only edit was a reorder as stock (TS, Python, C#). Python LONG coercion no longer rounds through float. Adds three shared extract-conformance cases. * fix: address pre-gate review findings (schema-valid pattern, caches, workspace walk, splitter dialect, constraint fallback) * refactor: simplify pass over the audit follow-up changes * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Fix XML child loss, eject whitespace, delimiter verify, JPY parse * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Drop C# source inference and Roslyn; tighten delimiter checks * no-mistakes(review): Pass inlinedAs only for partials; root ref renders * no-mistakes(review): Add csharp.namespace config, match pnpm globs, fix {{=}} crash * no-mistakes(review): Reject non-identifier derived C# namespace segments * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Resolve C# namespaces via package bindings; require Pydantic 2.8 * no-mistakes(review): Validate all package-binding namespaces as C# identifier paths * no-mistakes(review): Accept csharp.namespace in TS config schema; explicit handoff note * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Skip pnpm test dirs, reject C# keyword namespaces * no-mistakes(document): Fixed four stale doc facts from the regex-followups change: Java <filters> upgrade example (java.md and CHANGELOG), eject reorder wording (own-your-codegen.md), CSV numeric exemption (reporting.md). Edits verified by grep and diff stat. * no-mistakes(document): Fixed document-1: removed the stale single-Math.round claim from docs/ports/typescript-client.md, the parseCurrency header in currency.ts, and the currency-input.tsx comment. Doc and comment edits only; no remaining findings. * no-mistakes(lint): Update AGENTS.md and site payload fixture count to 421
1 parent 8a727b1 commit b36adc7

290 files changed

Lines changed: 6109 additions & 1242 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.githooks/leak-patterns.sh‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Shared leak-scan patterns, sourced by .githooks/pre-commit and .githooks/leak-scan.sh so the
2+
# local hook and the hosted `leak-scan` status can never disagree about what a leak is.
3+
# Names NO private project; private names come only from the untracked denylist.
4+
#
5+
# /Users/ is the macOS home shape. It was missing from leak-scan.sh until a rename-with-edits
6+
# slipped past both (audit finding 12).
7+
8+
# Generic, non-sensitive structural patterns (safe to commit):
9+
LEAK_PATTERNS='/home/[A-Za-z0-9._-]+/|/Users/[A-Za-z0-9._-]+/|~/Development'
10+
11+
# Legitimate matches to ignore (the published npm author email).
12+
LEAK_ALLOW='doug@dougmealing\.com'
13+
14+
# Diff filter: Added, Copied, Modified and Renamed (a `git mv` plus an edit is a rename).
15+
LEAK_DIFF_FILTER='ACMR'

‎.githooks/leak-scan.sh‎

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -15,13 +15,13 @@
1515
#
1616
set -uo pipefail
1717

18-
# Generic, non-sensitive structural patterns (safe to commit — name no project):
19-
PATTERNS='/home/[A-Za-z0-9._-]+/|~/Development'
20-
21-
# Legitimate matches to ignore (the published npm author email).
22-
ALLOW='doug@dougmealing\.com'
23-
2418
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || echo .)"
19+
# Patterns, allow-list and diff filter are shared with .githooks/pre-commit.
20+
# shellcheck source=leak-patterns.sh
21+
. "$ROOT/.githooks/leak-patterns.sh"
22+
PATTERNS="$LEAK_PATTERNS"
23+
ALLOW="$LEAK_ALLOW"
24+
2525
DENY_FILE="$(git config --get hooks.denyListPath 2>/dev/null || true)"
2626
[ -z "$DENY_FILE" ] && DENY_FILE="$ROOT/.githooks/deny-list.local"
2727
if [ -f "$DENY_FILE" ]; then
@@ -30,9 +30,9 @@ if [ -f "$DENY_FILE" ]; then
3030
fi
3131

3232
if [ -n "${1:-}" ]; then
33-
DIFF=$(git diff "$1...HEAD" -U0 --no-color --diff-filter=ACM 2>/dev/null)
33+
DIFF=$(git diff "$1...HEAD" -U0 --no-color --diff-filter="$LEAK_DIFF_FILTER" 2>/dev/null)
3434
else
35-
DIFF=$(git diff --cached --diff-filter=ACM -U0 --no-color 2>/dev/null)
35+
DIFF=$(git diff --cached --diff-filter="$LEAK_DIFF_FILTER" -U0 --no-color 2>/dev/null)
3636
fi
3737

3838
# Scan only newly-ADDED lines, attributed to their file; skip the hooks dir itself.

‎.githooks/pre-commit‎

Lines changed: 7 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -17,19 +17,17 @@
1717
#
1818
set -uo pipefail
1919

20-
# Generic, non-sensitive structural patterns — safe to commit (name no project):
21-
# /Users/ is the macOS home shape. It was missing until a plan doc authored on a Mac
22-
# committed an absolute developer home path that this hook scanned and passed.
23-
PATTERNS='/home/[A-Za-z0-9._-]+/|/Users/[A-Za-z0-9._-]+/|~/Development'
24-
25-
# Legitimate matches to ignore (e.g. the published npm author email).
26-
ALLOW='doug@dougmealing\.com'
20+
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || echo .)"
21+
# Patterns, allow-list and diff filter are shared with .githooks/leak-scan.sh.
22+
# shellcheck source=leak-patterns.sh
23+
. "$ROOT/.githooks/leak-patterns.sh"
24+
PATTERNS="$LEAK_PATTERNS"
25+
ALLOW="$LEAK_ALLOW"
2726

2827
# Merge in private-name patterns from a denylist. Source priority:
2928
# 1) git config hooks.denyListPath -> point at your private denylist file (kept in a private repo)
3029
# 2) .githooks/deny-list.local -> untracked local file or symlink
3130
# If neither resolves, only the generic structural patterns above are enforced.
32-
ROOT="$(git rev-parse --show-toplevel 2>/dev/null || echo .)"
3331
DENY_FILE="$(git config --get hooks.denyListPath 2>/dev/null || true)"
3432
[ -z "$DENY_FILE" ] && DENY_FILE="$ROOT/.githooks/deny-list.local"
3533
if [ -f "$DENY_FILE" ]; then
@@ -38,7 +36,7 @@ if [ -f "$DENY_FILE" ]; then
3836
fi
3937

4038
# Scan only newly-ADDED staged lines, attributed to their file; skip the hooks dir.
41-
hits=$(git diff --cached --diff-filter=ACM -U0 --no-color 2>/dev/null | awk '
39+
hits=$(git diff --cached --diff-filter="$LEAK_DIFF_FILTER" -U0 --no-color 2>/dev/null | awk '
4240
/^\+\+\+ / { f=$2; sub(/^b\//,"",f); next }
4341
/^\+/ && !/^\+\+\+/ { if (f !~ /^\.githooks\//) print f "\t" substr($0,2) }
4442
' | grep -Ei "$PATTERNS" | grep -vEi "$ALLOW")

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken.
7171
- **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres.
7272

7373
**Cross-port conformance corpora** (every port runs the shared corpus):
74-
- Metamodel: `fixtures/conformance/` (413 fixtures; 34 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
74+
- Metamodel: `fixtures/conformance/` (421 fixtures; 34 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
7575
- Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical.
7676
- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".)
7777
- API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived.

0 commit comments

Comments
 (0)