Unified Schema Registry: replace TS reflection + supplements + embedded widget templates

[ako]

Summary

Replace the current three-system metadata layer (TS-derived reflection data,
hand-maintained supplements.json, embedded widget templates) with a single
Schema Registry sourced empirically from mx dump-mpr (platform schemas)
and .mpk XML (widget schemas). Drives serialization, validation, inspection,
keyword dispatch, LSP, skills generation, and migration tooling end-to-end.

Full design: docs/11-proposals/UNIFIED_SCHEMA_REGISTRY.md

Supersedes #335 (kept for context).

Motivation

• Confusion: Today DATAGRID always picks the native builder, regardless
  of project version. On Mendix 11+ users want the pluggable variant.
• Stale data: TS reflection is frozen at 11.6; supplements.json is
  hand-maintained; embedded widget templates are one-version snapshots.
• Lost expressiveness: Widgets with object-list properties (Accordion
  groups, PopupMenu items, AreaChart series) are non-functional via the
  current pluggable engine.
• No version awareness: A script that writes valid BSON for 10.24 silently
  produces wrong BSON for 11.9.

Phases

• Phase 1 — mxcli schema extract platform

  • Wraps mx dump-mpr against blank .mpr per Mendix version
  • Outputs {version}-platform.json (types, storage names, list encodings,
    ref kinds, defaults, System module)
  • Embed ~15 LTS/MTS versions; downloadable for the rest
  • Validate by structural diff vs current reference/mendixmodellib/reflection-data/

• Phase 2 — mxcli schema extract widgets

  • Parse .mpk ZIP + {Widget}.xml into full property tree
  • Capture object-list sub-property trees
  • Cache at ~/.mxcli/widget-schemas/{widgetId}@{version}.json
  • Drop sdk/widgets/templates/, sdk/widgets/augment.go, .mxcli/widgets/*.def.json

• Phase 3 — Registry runtime (mdl/backend/schema/)

  • Registry, TypeSchema, PropertySchema, WidgetSchema, KeywordMapping
  • Wire writer (completeness check) + reader (tolerant parse)
  • Drop supplements.json, reference/mendixmodellib/reflection-data/,
    sdk/mpr/system_module.go
  • CLI: mxcli schema list/show/diff
  • Catalog tables: schema_types, schema_properties, schema_keywords

• Phase 4 — Native/pluggable dispatch + object lists

  • Replace hardcoded case \"datagrid\" etc. with Registry.LookupKeyword(kw, version)
  • Grammar/visitor/executor support for object-list child block syntax
    (group, item, series, marker from widget schema)
  • Migrate static keyword builders to thin wrappers over registry

• Phase 5 — Workflow integration

  • mxcli init runs Phases 1+2 extraction
  • refresh catalog re-runs Phase 2 on .mpk changes
  • mxcli check validates property bindings, version compatibility, BSON drift
  • mxcli check --post-migration, mxcli upgrade pages, mxcli upgrade widgets
  • LSP wiring: completion, hover, diagnostics from registry
  • Skills regeneration from registry
  • mxcli syntax topics gain "see also: `schema show `" links

Acceptance criteria

• The same MDL script produces native DATAGRID BSON on Mendix 10.24 and pluggable
  Datagrid BSON on 11.9, both opening cleanly in Studio Pro.
• Accordion group, PopupMenu item, Maps marker all work via MDL.
• mxcli check rejects writes that target properties unavailable in the project's
  Mendix version, with actionable hints.
• mxcli widget upgrade <widget> auto-fixes additive-safe drift, flags everything else.
• All sdk/mpr/ writer paths consume Registry; no subsystem reads reflection-data
  or supplements.json directly.

Non-goals

• Replicating Studio Pro's project-upgrade transformations
• Pinning a project to multiple Mendix versions simultaneously
• Predicting changes for unreleased Mendix versions

Risks

• mx dump-mpr JSON format stability across Mendix versions
• Widget XML schema drift across versions
• Performance of registry construction at every CLI invocation (mitigation: lazy load + binary cache)

🤖 Generated with Claude Code

[github-actions]

Issue Triage

Summary:
The issue proposes a Unified Schema Registry to replace the current three‑system metadata layer, driving serialization, validation, inspection, and other tooling end‑to‑end.

Checklist for feature request

Required info	Present?	Notes
Use case – why the feature is needed, what problem it solves	✅	Detailed motivation (confusion, stale data, lost expressiveness, no version awareness).
Proposed syntax or behavior (if applicable)	✅	Five‑phase plan with concrete CLI commands, registry runtime, dispatch changes, workflow integration, and acceptance criteria.
Mendix version relevance (if version‑specific)	✅	References specific versions (10.24, 11.9) and discusses version‑aware behavior across LTS/MTS releases.

Conclusion:
All required information for a feature request is present. Thank you for the thorough proposal! 🙌

---

Automated triage via OpenRouter — workflow source

[ako]

Refinement: dual-stack widgets

Updated the proposal to handle a case the original draft didn't cover: Mendix
project upgrades from 10 to 11 do not auto-convert native (dojo) widgets
to their pluggable (React) replacements. Studio Pro changes the project
version, but the widget instances stay on the legacy stack until the user
manually replaces them. Migrated projects therefore routinely have both
stacks coexisting on the same page.

This breaks the version-range-only dispatch model from the original proposal
(DATAGRID @ 10.x → native, DATAGRID @ 11.x → pluggable). On a migrated
11.9 project, both forms can be valid simultaneously.

Resolution

Distinct MDL keywords per stack, with a version-aware default for the
recommended stack:

Legacy native (dojo)	Modern pluggable (React)
LEGACYDATAGRID → Forms$DataGrid	DATAGRID → com.mendix.widget.web.datagrid.Datagrid
LEGACYLISTVIEW → Forms$ListView	GALLERY → com.mendix.widget.custom.gallery.Gallery
LEGACYDROPDOWN → Forms$DropDown	COMBOBOX → com.mendix.widget.web.combobox.Combobox

DESCRIBE PAGE round-trips each widget to the keyword matching its actual
BSON stack, so editing mixed projects via MDL stays unambiguous. The
DATAGRID keyword on Mendix 11+ defaults to pluggable; on 10.x it's the
only option (native).

mxcli check --post-migration becomes more useful

Now lists every legacy-stack widget on the project, giving users a visible
migration to-do rather than discovering legacy grids page by page:

Project on Mendix 11.9 — 14 widgets still on legacy (dojo) stack:
  Pages/Customers     LEGACYDATAGRID dgCustomers   (consider DATAGRID)
  Pages/Orders        LEGACYDROPDOWN ddStatus      (consider COMBOBOX)
  Pages/OrderDetail   LEGACYLISTVIEW lvLines       (consider GALLERY)
  ...

Auto-conversion is a non-goal

Converting Forms$DataGrid BSON to com.mendix.widget.web.datagrid.Datagrid
BSON requires widget-author-specific knowledge of property mapping between
the two stacks (different runtime, different schemas). mxcli surfaces legacy
widgets but doesn't rewrite them — users convert in Studio Pro one widget
at a time.

Diff in commit edb1554f on the docs branch (will land in the same docs PR
as the original proposal).

🤖 Generated with Claude Code

[ako]

Phasing revision (rev 2)

Two corrections after a critique pass:

1. mx dump-mpr framing was overstated

The earlier draft framed mx dump-mpr as risky/unstable. That's wrong — it's the
Mendix-shipped CLI export of the same data Studio Pro reads. Equivalent in authority
to a hypothetical Mendix-published schema package; the only difference is the
invocation step. Withdrawing the brittleness framing.

2. Phase reordering — widget improvements first

The widget extraction half (mxcli widget init → .mxcli/widgets/*.def.json) is
already shipped. Work is maturation, not greenfield. Phases now front-load
widget-side improvements and defer platform-schema work using build-time codegen
instead of a runtime registry.

Old phasing	Revised phasing	Visible value at boundary
1. schema extract platform	1. Object list support (extends existing .def.json format)	Accordion groups, PopupMenu items, Maps markers work in MDL
2. schema extract widgets	2. Dual-stack dispatch table (small policy data file)	DATAGRID does the right thing per Mendix version; mixed migrated projects round-trip
3. Registry runtime	3. Workflow integration (widget side)	init/refresh/check/lsp/skills threaded around existing widget engine
4. Native/pluggable + object lists	4. Platform schema codegen (mx dump-mpr → existing cmd/codegen/)	Compile-time-typed schema lookups; replaces system_module.go
5. Workflow integration	5. Drop supplements.json + cross-version inspection	Zero hand-maintained schema data; migration commands end-to-end

Key implementation change: platform schemas become codegen output, not a runtime
registry. Reuses the existing cmd/codegen/main.go pipeline; format changes surface
as build failures rather than runtime errors. The widget half stays runtime
(per-project on-disk JSON) since .mpk files vary per project.

Other corrections

• Architecture diagram revised to show the two halves' distinct lifecycles
  (compile-time codegen vs runtime extraction)
• Trade-offs table adds an explicit decision: platform schemas via codegen
• Performance / format-stability open questions softened to reflect the codegen approach

Diff in commit 279aa555 on the issues branch (will land in the same docs PR
as the original proposal).

🤖 Generated with Claude Code