Skip to content

The undeclared-field door sits in FRONT of the hooks, so a key a beforeInsert hook writes has no door at all — and the drivers then disagree (memory stores it, SQL throws a raw statement error) #13657

Description

@os-trump

Measured on @objectstack/* 17.1.0, booting the shipped HotCRM stack on three drivers. Filed from objectstack-ai/hotcrm#1200, whose deliverable was explicitly measurement + one upstream question rather than a fix in the app.

The good half first: #8682 / #8738 worked

#8682 (insert) and #8738 (update) put a declared-field door in, and PR #8737 moved it ahead of hooks and ahead of statement construction. That reproduces exactly as intended. A key the CALLER supplies is refused before anything happens, identically on all three drivers:

insert crm_opportunity_line_item { crm_opportunity, crm_product, quantity: 1, unit_price: 100, tax_rate: 10 }
  -> INVALID_FIELD / 400 / Unknown field 'tax_rate' on object 'crm_opportunity_line_item'

Same for update, and same for a plain misspelling (unit_pric). Controls both ways: the same row without the key is accepted, and the same tax_rate key sent to the twin object crm_quote_line_item, which DOES declare it, is accepted and stored as 10. So the door is real and it is about declaration, not about a name.

The gap: the door's new position leaves the hook path undefended

undeclaredWriteFieldErrors(object, schema, rows) is called as the first statement inside executeWithMiddleware in both ObjectQLEngine.insert and .update (@objectstack/objectql/dist/core.js), and the beforeInsert / beforeUpdate hook contexts are built and dispatched after it. Fixing "the door ran too late" by moving it in front of the hooks means a key the hooks themselves assign is never checked against the schema at all. Whatever the hook wrote goes straight to the driver.

That is not a hypothetical seam. It is how metadata apps are written: every hook in the HotCRM exemplar assigns by name (input.list_price = …), and a hook is exactly where a misspelling is least likely to be caught by review.

And there the three drivers do three different things

Probe: a test-only beforeInsert hook on crm_opportunity_line_item that sets ctx.input.tax_rate = 10, a field the object does not declare. Caller sends nothing undeclared.

driver outcome error code status row afterwards
memory ACCEPTED tax_rate: 10 stored and returned on read; stored keys created_at, crm_opportunity, crm_product, description, discount, id, list_price, quantity, tax_rate, total_price, unit_price, updated_at
sqlite (driver-sql / knex) refused SQLITE_ERROR (none) no row
sqlite-wasm refused (none — a bare Error) (none) no row

Three points, in order of weight:

  1. The behaviour differs by driver. The same metadata and the same hook code mean different things on two deployments, and nothing in the app can tell which one it is running on. That is worse than either consistent answer: a customer on memory/mongodb-shaped storage silently grows a shadow column, a customer on SQL gets a hard write failure, from one identical app.
  2. Neither refusal is an ADR-0112 envelope. No code on one, a backend-specific SQLITE_ERROR on the other, no status on either — so no caller can handle this by shape, and the two SQL drivers do not even agree with each other. Contrast the caller path, which answers a clean INVALID_FIELD / 400.
  3. On memory the security corollary from hotcrm#1200 is live. Field-level permissions are fieldPermissions: Record FIELDNAME, { readable, editable } — keyed by field name against the object's declared field set. A key the object does not declare can have no entry, so a hook-written undeclared value sits outside allowEdit and field-level security by construction, and no view, formula, index or permission knows it exists. The only symptom is a correctly-spelled field that never seems to update.

Secondarily: #8682's Half B exposure survives on this path. Both SQL refusals carry the full bound INSERT statement with its values in the error message — the caller path no longer does this, but the hook path still does.

Is there a switch? No — that is reading 2 of the hotcrm card

Checked against the shipped strict schemas rather than guessed:

  • ObjectSchema (@objectstack/spec/data, .strict()) — full top-level key set enumerated; no strict / schemaless / allowUnknownFields / additionalFields key exists. enable.* (trackHistory, searchable, apiEnabled, apiMethods, files, feeds, activities, clone) and protection.* (lock, reason, docsUrl) are unrelated.
  • DatasourceSchema (.strict()) — name, label, driver, config, pool, ssl, description, active, autoConnect, schemaMode, external, origin plus the _lock* / _provenance family. schemaMode (managed / external / validate-only) is DDL ownership per ADR-0015, and external.validation.onMismatch is schema-DRIFT posture; neither is a per-write key check.
  • No env switch gates the guard either. @objectstack/objectql's env surface is OS_ALLOW_DRIVER_CONNECT_FAILURE, OS_ALLOW_LAX_MEDIA_VALUES, OS_ALLOW_LAX_VALUE_SHAPES, OS_DATABASE_URL, OS_DATA_VALUE_SHAPE_STRICT_ENABLED, OS_INLINE_SEED_BUDGET_MS, OS_METADATA_COLLISION, OS_REGISTRY_LOG, OS_SEARCH_PINYIN_ENABLED, OS_TENANCY_POSTURE — the value-shape family is a sibling strictness posture with a real flag and a documented loosen escape. The KEY set has no equivalent posture on the hook path, in either direction.

So: on the caller path strict is the unconditional default and needs no switch, and on the hook path there is no switch that would make it strict.

Why this is a gap rather than schemaless-by-design

The question hotcrm#1200 raised is whether an undeclared write is intended. The argument its author gave is the one that seems decisive, so it is carried over verbatim in substance: ADR-0104 D3 wave 2 moved accept / maxSize from browser-only to server-enforced on the grounds that a constraint that exists only client-side is not a constraint. A field list enforced on the caller path but not on the hook path is the same shape one layer in — the constraint holds against the actor least likely to be wrong (an external caller, whose payload is already schema-checked at the REST boundary) and does not hold against the actor most likely to be wrong (app code assigning by name in a hook body).

Suggested direction — noting the ordering constraint #8682 was solving

The obvious repair is to re-check after the before* hooks as well, but #8682's whole point was that work must not happen before a refusal. Both can hold: keep the existing pre-hook door exactly where it is (it still refuses caller payloads before an auto-number is consumed), and add a post-hook, pre-statement check on the same declared-field set, so a hook-written key is refused with the same INVALID_FIELD / 400 envelope on every driver instead of being stored by one and crashed by another. If a schemaless posture is wanted for memory-family stores, it should be a declared posture with a name, not an accident of which driver a deployment happens to run.

Reproduce

test/undeclared-key-probe.test.ts in objectstack-ai/hotcrm (branch claude/issue-1200-undeclared-key-probe) is the runnable probe — kernel boot per driver via DefaultDatasourcePlugin + AppPlugin(stack, undefined, { skipSeedData: true }), both halves asserted, with the two controls described above.

Back-link: objectstack-ai/hotcrm#1200. Filed unassigned and ungraded — this repo's triage seat owns domain:* and type.

Metadata

Metadata

Assignees

Type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions