From 7bd91e74320c489f2d7a24f79e0a98cd0d38aac6 Mon Sep 17 00:00:00 2001 From: delchev Date: Tue, 8 Sep 2026 11:02:07 +0300 Subject: [PATCH] intent: the posting rewrite is one transaction - a refused line leaves the previous post standing (#7132) The #7071 rewrite path deleted every stale item row, updated the header and then saved the derived lines, each store call its own transaction. A failure on the Nth line - now easy to reach, since #7069's required-value check throws a ValidationException from the item save - left the journal entry with a header and a partial set of lines. Unlike the half-post case there is no second event to self-heal from: the source has already reached its moment and raises nothing further, so the ledger ended up WORSE than the stale but balanced state the rewrite set out to fix. The handler's whole write phase now runs in one UnitOfWork.run(...) - the rows a rewrite replaces, the header, every derived line - as Generate.java.template has since #7069: either the post says what the source derives now, or it still says what it said before. Across STEPS the consistency model is unchanged and deliberately not transactional; a bad post is unwound by a correcting entry (reverses:), not a rollback. Mechanically this only hoists currentItems out of the refresh branch (both branches assign it once, so the block can close over it) and moves the delete/update/save into the block. The decision half - the cell-by-cell comparison, the redelivery no-op, the amendable guard - stays outside: it only reads, and it returns before anything is written. Covered by IntentEngineIT and IntentEmissionCoverageIT, both asserting by POSITION that the delete, the header write and the line writes sit inside the block - a header written outside it with the lines inside would still "mention UnitOfWork". IntentEmissionCoverageIT additionally compiles the generated handler with the real javac batch. Fixes #7132 Co-Authored-By: Claude Opus 5 --- .claude/docs/intent-layer.md | 2 +- components/engine/engine-intent/CLAUDE.md | 2 +- .../events/Posting.java.template | 51 ++++++++++++------- .../tests/api/IntentEmissionCoverageIT.java | 13 +++++ .../integration/tests/api/IntentEngineIT.java | 17 +++++-- 5 files changed, 61 insertions(+), 24 deletions(-) diff --git a/.claude/docs/intent-layer.md b/.claude/docs/intent-layer.md index c79a6cf1cfd..fd4a6e98614 100644 --- a/.claude/docs/intent-layer.md +++ b/.claude/docs/intent-layer.md @@ -34,7 +34,7 @@ A single `app.intent` YAML file at a project root is the source of truth one alt **A create-from is not offered twice (`fromStatus:`, [#7068](https://github.com/eclipse-dirigible/dirigible/issues/7068)):** a `generates:` with a `sourceStatus:` completion hook flipped its source once the target existed and then went on offering the same button on the flipped record - and answering the same endpoint 200 - so a second click minted a **second document**: a proforma already INVOICED produced a second invoice, in the customer's hands. The hook declared what "already done" looks like; nothing consulted it. A create-from now carries a from-status guard resolved ONCE and fed to both halves of the action: the generated `run()` refuses with **409** before anything is created, and the contributed action descriptor carries the same guard so the shared `customActions` store stops OFFERING the click on a record it would refuse (`getActions(view, type, record)` takes the record the view already has). Two shapes: `fromStatus: [...]` is the explicit allow-list - the `from:` of a `transitions:` entry, spelled differently only because `from:` on a create-from already names the source ENTITY - and absent it a declared `sourceStatus` IMPLIES the deny-list of exactly that status, so a model that already carries the defect is fixed with no authoring change. The guard is on the CLICK: an event-driven create-from keeps its own at-most-once back-reference guard and qualifies its moment with `event.when`, so `fromStatus` on an event-only rule is refused at parse rather than silently ignored - as are a `page` scope, a source with no `function: EntityStatus` relation, and an allow-list containing the `sourceStatus` the action itself writes. -**An amended source rewrites its posting ([#7071](https://github.com/eclipse-dirigible/dirigible/issues/7071)):** the amend path (Confirm → Reject → edit the lines → Issue again) raises a `postings:` trigger a SECOND time, and the old idempotency test - an existing post whose item count reached the derived one - read that as "already posted", so the journal entry silently kept the amounts of the previous issue while the invoice it references had moved on. No second entry (right), a ledger short by the difference (wrong), and nothing anywhere said so. The generated handler now derives the WHOLE content first and compares it with what the post carries: identical is a redelivery (no-op), different is either a half-post to complete or an amendment to REWRITE the post from - header assignments re-applied, items replaced, never a second document. The rewrite stops where the created document's own lifecycle says someone has taken it over: it is rewritable only while its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (with no status lifecycle there is nothing to act on, so it is always rewritable), and past that the divergence is logged naming both documents and left to a correcting entry - `reverses:` - rather than overwritten behind the accountant's back. The comparison is order-insensitive over every cell the item rows assign, and numbers compare by value so a rescaled amount is not a change. +**An amended source rewrites its posting ([#7071](https://github.com/eclipse-dirigible/dirigible/issues/7071)):** the amend path (Confirm → Reject → edit the lines → Issue again) raises a `postings:` trigger a SECOND time, and the old idempotency test - an existing post whose item count reached the derived one - read that as "already posted", so the journal entry silently kept the amounts of the previous issue while the invoice it references had moved on. No second entry (right), a ledger short by the difference (wrong), and nothing anywhere said so. The generated handler now derives the WHOLE content first and compares it with what the post carries: identical is a redelivery (no-op), different is either a half-post to complete or an amendment to REWRITE the post from - header assignments re-applied, items replaced, never a second document. The rewrite stops where the created document's own lifecycle says someone has taken it over: it is rewritable only while its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (with no status lifecycle there is nothing to act on, so it is always rewritable), and past that the divergence is logged naming both documents and left to a correcting entry - `reverses:` - rather than overwritten behind the accountant's back. The comparison is order-insensitive over every cell the item rows assign, and numbers compare by value so a rescaled amount is not a change. The rewrite is ONE transaction ([#7132](https://github.com/eclipse-dirigible/dirigible/issues/7132)) - the stale rows it replaces, the header and every derived line share a `UnitOfWork`, because a line a validation refuses halfway would leave a header with a partial line set, and unlike the half-post case no second event ever comes to repair it: worse than the stale but balanced post the rewrite set out to fix. **Event-driven create-from (`generates` + `event:`, [#6711](https://github.com/eclipse-dirigible/dirigible/issues/6711)):** a `generates` entry may declare `event: { onTransition: , when: " == " }` (guard mandatory) or `{ onCreate: }` and mint the follow-up **document — header AND items** by itself when the source reaches a state, instead of waiting for the button (`posts` is event-driven but emits flat rows and cannot reference the new header). The `map` entry copying the source's key IS the back-reference and therefore the **at-most-once** guard, derived rather than declared twice; the button is dropped unless `button: true`, and both triggers share ONE generated create-from (a new `GenerateOnEvent.java.template` listener calls `Generate.create(id)` and carries no mapping of its own). Details in the engine-intent guide's `event:` bullet. diff --git a/components/engine/engine-intent/CLAUDE.md b/components/engine/engine-intent/CLAUDE.md index f55277fe2ba..066c8c9e33c 100644 --- a/components/engine/engine-intent/CLAUDE.md +++ b/components/engine/engine-intent/CLAUDE.md @@ -418,7 +418,7 @@ Semantics worth knowing: - **`init: ` on a to-one relation = the FK's DB-level default (`RelationIntent.init` → `dataDefaultValue` on the FK property, in both `relationProperty` and `crossModelRelationProperty`).** The relation analogue of a field's `defaultValue`; a new row gets this FK on insert when the column is left unset (e.g. `Status` defaults to the DRAFT seed, `PaymentMethod` to Bank, `SentMethod` to E-mail). **Use `init` for an INITIAL status, never a process step.** A DB default is applied at insert — atomic, no ordering to reason about — while a start-step setter is extra moving parts for the same effect. (Since #7104 the generated repository also assigns the default itself, before the create-time calculations run, so a calculation reading a defaulted column no longer sees the null the DB was about to fill — the DB DEFAULT remains for every write that does not go through the repository.) (Historical note: a start-step `setRelationField` also used to be *clobbered* by the trigger's `ProcessId` write-back, which was a full-row `updateWithoutEvent` merge of a stale snapshot — confirmed live: the invoice reached the Approve task but `Status` stayed null. The trigger now persists `ProcessId` via the targeted single-column `repository.updateProperty(...)` (SDK `JavaRepository`/`JavaEntityStore`), and a minted business key the same way before the start, so that race is gone; `init:` remains the right modeling for an initial status.) `setRelationField` is for *transitions* (after a user task / on a decision branch). - **`trigger: { onCreate|onUpdate|onDelete: , when: "" }` starts the process on the named `` lifecycle event** - fully wired (Java). Any of the three events is supported: `onCreate` binds the entity's base topic, `onUpdate`/`onDelete` the `-updated`/`-deleted` topics the Java DAO publishes (`TriggerSupport` + `EventBinding`); an optional `when` guard (a single `field ==|!= literal`, via `NotificationSupport.guard`) gates `Process.start`. Three parts: (1) the parser validates at most one event kind and that the target is a declared entity; (2) the EDM generator adds a `ProcessId` back-reference property (VARCHAR) plus the per-process `ProcessIds` stamps column (VARCHAR, `Process=instanceId` pairs) to that entity and a `triggers` collection to the `.model` (`TriggerSupport` + `EdmIntentGenerator.buildTriggers`); (3) the **`template-application-events-java`** template (intent-driven, like the other language templates) reads that `triggers` collection and emits one **`gen/events//Trigger.java`** per trigger - a client-Java self-describing `MessageHandler` (a `@Component` whose `destination()` is the entity's per-operation topic via `topicSuffix` and whose `kind()` is `TOPIC`) that loads the entity, applies the `when` guard, calls `Process.start(, businessKey, )`, and writes the instance id back to `ProcessIds` against its own process name plus `ProcessId` (so THIS process starts at most once - one `ProcessId` cannot say WHICH process ran, and reading it as "some process ran" silently skipped every follow-up flow on an already-stamped record, #6862). **The write-back is crash-safe by construction (#6815):** the per-process stamp in `ProcessIds` IS the at-most-once guard while the start and the write-back commit independently, so everything that can precede the start does — the minted business key is persisted first, and every process variable (the `__entityUrl`/`__entityId` locators, the FK locators, `__personalUser`) rides the start payload instead of a post-start `setVariable` (all are known up front, and a wait-state-less process finishes inside `start`, where a `setVariable` would then throw). The one remaining post-start step is the targeted `updateProperties` of both process columns at once (a record carrying one without the other is either invisible to the task UI or blocked from ever starting the flow again), which a `checks:` gate can no longer refuse (the generated repository runs `enforceChecks` only for a write that touches an **authored** column - a gate has no opinion about which process handles the document, and by then the instance is running), a swallowed start (`null` id) is logged rather than written, and if the write still does not land — the row was deleted meanwhile, or it threw — the instance is **cancelled** (`Process.cancel`) and the failure re-thrown, rather than left running with nothing pointing at it. The Java DAO template (`template-application-dao-java`) now publishes the create event (`Producer.sendToTopic('${projectName}-${perspectiveName}-${name}', json)`) the way the TS DAO does - that's the topic the handler binds to. `gen/events/` (the `` segment = the sanitized intent name, `IntentNaming.javaModule`) is a sibling of `gen/`, so it survives the per-model regeneration wipe. The events template iterates the model's `triggers` via a new **`triggers` collection case in the generation pipeline's `ModelGenerator`** (the engine's collection switch is hardcoded; the case has its own loop because triggers are not entity-shaped). The BPM **business key** defaults to the entity's primary key but is **configurable**: `trigger: { ..., businessKey: }` names which trigger-entity field becomes the started instance's business key (the listener still loads the entity by its PK via `findById`; only the business key differs — a separate `businessKeyProperty` in `.glue`). An optional `businessKeyStrategy: timestamp` mints a `yyyyMMddHHmmss` value into that field when it is blank and persists it via the listener's existing update — the simple "for now" generator and the **extension point** for richer pluggable number generators later (sequential, zero-padded, config-prefixed invoice numbers); the parser validates the field exists, the strategy is supported, and (for `timestamp`) the field is `string`/`text`. `TriggerSupport.triggerBusinessKey`/`triggerBusinessKeyStrategy` read them; `GlueIntentGenerator` emits `businessKeyProperty` + `generateBusinessKey`; `Trigger.java.template` renders the mint-if-blank block. `onSchedule` is still unmodelled. **Casing subtlety in the generated handler:** its `import gen..data..{Entity,Repository}` must use the **lowercased** Java package segment (`javaPerspective` = `sanitizeJavaIdentifier(perspective)`, matching the DAO/entity templates' `javaPerspectiveName` folder), while the `destination()` topic (`"--"`) keeps the **raw** perspective so it matches the topic the DAO publishes to (`${projectName}-${perspectiveName}-${name}`). The `triggers` collection case in the pipeline supplies both (`javaPerspective` for the import, `perspective` for the topic). Using the raw perspective in the import compiled on macOS (case-insensitive FS) but failed `javac` with "package gen.x.data.Member does not exist" because the entity files declare the lowercased package. - **`dependsOn` on a to-one relation or a field = the EDM Depends-On feature (cascading dropdowns + auto-populated fields).** `dependsOn: { relation: , valueFrom?: , filterBy?: }` — the widget reacts to the sibling trigger: the generated form loads the trigger's selected record, reads `valueFrom` (default: the trigger target's PK), then a **relation** re-filters its dropdown options where its own target's `filterBy` (default: that target's PK) equals the value (`POST /search` with an EQ condition; a single remaining option auto-selects), while a **field** copies the value (auto-population; `valueFrom` mandatory, `filterBy` rejected). Emitted by `EdmIntentGenerator.putDependsOn` as the four scalar `widgetDependsOn*` property attributes the AngularJS stacks already consume (so those work for free); the Harmonia runtime was added in the same pass (`form-page.js.template` per-property watcher + `applyDependsOn` methods covering manage/master-detail/allocation forms; `document-page.js.template` header watchers + a generic metadata-driven `applyDraftDependsOn` for the line-item dialog off `detail-register.js.template`'s `editColumns[].dependsOn`, with filtered options in a separate `draftOptions` store so the items table's label resolution keeps the full set; `ModelParameterProcessor` precomputes `widgetDependsOnControllerUrl` from the trigger sibling). `valueFrom`/`filterBy` use the target's **authored** property names (field lower-camel / relation as declared); same-model references are parse-validated, cross-model ones generation-validated against the resolved owner model (`CrossModelSupport.TargetInfo.propertyNames`). A `documentStatus` relation can neither declare nor trigger a dependsOn. Canonical cases (the `codbex-sample-model-depends-on` set): Country→City cascade (`filterBy` only), Product→UoM narrow-to-referenced (`valueFrom` only), Product→price auto-populate (field). **Conditional auto-populate (#6358):** a FIELD's `valueFrom` may be `{ by: , cases: { : }, default?: }` — the copied trigger-target property is picked by a classifier resolved from the `by` path (own property / one-hop `.` / a path starting at the composition parent relation = the open document header). Parser `validateConditionalValueFrom` (shape, path segments, case/default properties against the trigger target); EDM emits `widgetDependsOnValueBy` (+`ByHeader`/`ByHeaderEntity`/`ByEntity` for the hop fetch), `widgetDependsOnValueCases` (JSON string, PascalCased properties), `widgetDependsOnValueDefault`, and NO `widgetDependsOnValueFrom`; `ModelParameterProcessor` derives `widgetDependsOnValueByUrl` (the hop record's controller); Harmonia consumes it via `resolveDependsOnSource` (document page: dialog + header form; `resolveDependsOnSource` on the manage form) — Harmonia-only, the AngularJS `#if` guards skip it (no `ValueFrom`). Editor round-trip: the six attrs are in `model.js`/`serializer.js` (no dialog UI - intent is the source). **Header-mediated trigger (#6358, the issue's other half):** `relation: .
` on a document ITEM field (`relation: SalesInvoice.Customer, valueFrom: standardDiscount`) makes the line default from a record the open DOCUMENT points at instead of one of the line's own relations - the canonical case being a line discount defaulting from the header partner's terms. Parser `validateHeaderMediatedDependsOn` (fields only - a header selection has no option list to cascade, so `valueFrom` is mandatory and `filterBy` rejected; the first segment must be the composition parent, the second a to-one of the header, and `valueFrom` resolves against THAT relation's target). `putDependsOn` resolves the trigger through the header and adds `widgetDependsOnHeader` + `widgetDependsOnHeaderEntity`; `ModelParameterProcessor` resolves `widgetDependsOnControllerUrl` on the HEADER entity (the trigger is not a property of the item). Harmonia: `detail-register` emits `dependsOn.header`, and `document-page` gains `applyHeaderDependsOnToDraft` - called on a CREATE draft open and from a `form.` watcher while the dialog is open, never on an edit draft (the stored value may be a deliberate override); `applyDraftDependsOn`/the draft watchers explicitly skip header columns so a same-named row column cannot drive them. **Every sibling-assuming stack is guarded** (`&& !$property.widgetDependsOnHeader` in the four `-java`/`-v2`/legacy AngularJS controller templates and the Harmonia `form-page`) - without it they emit a watcher on `entity.` / `this.form.` that does not exist on the item. Composable with the conditional `valueFrom`. -- **`postings:` (top-level) = declarative posting (source-document status → generated local document + computed items).** The accounting "documents → ledger" capability, generalized (spike-derived; see the driving suite's spike findings). `PostingIntent` + parser `validatePostings` (creates = local document owning a composition items child; backReference = its to-one to the source, the at-most-once guard; event trigger `onTransition` with a mandatory `when: " == "` status guard, or `onCreate` for a source with NO status lifecycle - a booked payment - binding the `-created` topic with the `when` guard optional (#6421); item cells = `rule()` refs into a single-selector rule entity or Calc arithmetic over the source; row `when: ==|!= `). `GlueIntentGenerator.buildPostings` pre-renders EVERYTHING as Java expressions (the expansions convention — the template stays shape-only): topic + re-load coordinates via `CrossModelSupport`, guard, header assignments (copy / literal / `{placeholder}` concat), `ruleRow.` refs, `Calc.eval("", source, )` amounts with the scale from the LOCAL item field, null-safe Calc row guards. `postings` glue collection → the pipeline's collection case (source gen folder = sanitized model alias, topic keeps the RAW perspective) → `Posting.java.template`: a `MessageHandler` on `---transitioned` (#6220's channel) that re-loads the source by id (the payload lacks later-step data — the stamped number), guards, resolves the rule row (missing row / null referenced column → SKIP, the unposted worklist), and writes target + items through the repositories — so numbering / status `init:` / `checks:` fire on the created document. **Idempotent + resumable + amendable, not transactional** (the cloud-native consistency model — there is NO cross-step DB rollback; each write commits on its own): the handler derives the WHOLE content first and compares it with the post the back-reference finds — identical is a redelivery (no-op), different is either a HALF-post (an item write failed after the target was saved) to complete or an AMENDED source to rewrite from. The amendment half is #7071: the amend path (Confirm → Reject → edit the lines → Issue again) raises the SAME moment a second time, and the old `item count ≥ expectedItems` test read that as "already posted", so the entry kept the amounts of the previous issue while the document it references had moved on — no second entry (right) and a ledger 60.00 short (wrong), silently. Now the existing post is REWRITTEN in place (header assignments re-applied through `update`, items replaced) — but only while nobody has acted on the created document, which the posting itself defines: its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (a target with no status lifecycle is always rewritable, one whose status is declared without an `init:` is rewritable while still empty). Once it has moved, the divergence is LOGGED naming both documents and the entry is left alone — unwinding a posted entry is a correcting entry's job (`reverses:`), not a silent overwrite. The comparison is order-insensitive (row order is not a query guarantee) over the union of every cell the item rows assign (`itemComparedProps`), and numbers compare by VALUE so a rescaled amount is not a change; `amendableGuard`/`itemComparedProps` are pre-rendered into the glue like everything else, and both default to the pre-#7071 behaviour when a `.glue` predates them. Concurrent-redelivery de-duplication is best-effort (a check-then-act on the back-reference) until a real UNIQUE key on the back-reference lands with schema constraint emission. Storno/negation mode LANDED as **`reverses:`** (paired with the `transitions:` void primitive - the "void-document event" is a transition into the void status): a reversal posting inherits creates/backReference/rule/map/items from the reversed sibling, negates every item amount expression on the SAME side (`Calc.eval("-()", ...)` - red storno), locates the original through the empty `storno:` self-link (none -> fail-soft skip), stamps the link on its creation, and both handlers' idempotency guards discriminate by that link (reversal counts linked rows, the sibling counts unlinked ones - `stornoProperty`/`stornoFilterProperty` in the glue). The explicit manual Reverse action (no source void) remains a follow-up. Compensation, not a transaction, is how a bad post is unwound. +- **`postings:` (top-level) = declarative posting (source-document status → generated local document + computed items).** The accounting "documents → ledger" capability, generalized (spike-derived; see the driving suite's spike findings). `PostingIntent` + parser `validatePostings` (creates = local document owning a composition items child; backReference = its to-one to the source, the at-most-once guard; event trigger `onTransition` with a mandatory `when: " == "` status guard, or `onCreate` for a source with NO status lifecycle - a booked payment - binding the `-created` topic with the `when` guard optional (#6421); item cells = `rule()` refs into a single-selector rule entity or Calc arithmetic over the source; row `when: ==|!= `). `GlueIntentGenerator.buildPostings` pre-renders EVERYTHING as Java expressions (the expansions convention — the template stays shape-only): topic + re-load coordinates via `CrossModelSupport`, guard, header assignments (copy / literal / `{placeholder}` concat), `ruleRow.` refs, `Calc.eval("", source, )` amounts with the scale from the LOCAL item field, null-safe Calc row guards. `postings` glue collection → the pipeline's collection case (source gen folder = sanitized model alias, topic keeps the RAW perspective) → `Posting.java.template`: a `MessageHandler` on `---transitioned` (#6220's channel) that re-loads the source by id (the payload lacks later-step data — the stamped number), guards, resolves the rule row (missing row / null referenced column → SKIP, the unposted worklist), and writes target + items through the repositories — so numbering / status `init:` / `checks:` fire on the created document. **Idempotent + resumable + amendable, and the post itself is ONE transaction** — the handler's own writes (the stale rows a rewrite replaces, the header, every derived line) share a `UnitOfWork`, so a line the item repository refuses leaves the previous post standing instead of a header with a partial line set (#7132: unlike the half-post case there is no second event to self-heal from, so a partial rewrite ends up worse than the stale but balanced post it set out to fix). Across STEPS the model is unchanged and deliberately not transactional — the source's own commit, this handler's post and a reversal are separate events, and a bad post is unwound by a correcting entry, not a rollback: the handler derives the WHOLE content first and compares it with the post the back-reference finds — identical is a redelivery (no-op), different is either a HALF-post (an item write failed after the target was saved) to complete or an AMENDED source to rewrite from. The amendment half is #7071: the amend path (Confirm → Reject → edit the lines → Issue again) raises the SAME moment a second time, and the old `item count ≥ expectedItems` test read that as "already posted", so the entry kept the amounts of the previous issue while the document it references had moved on — no second entry (right) and a ledger 60.00 short (wrong), silently. Now the existing post is REWRITTEN in place (header assignments re-applied through `update`, items replaced) — but only while nobody has acted on the created document, which the posting itself defines: its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (a target with no status lifecycle is always rewritable, one whose status is declared without an `init:` is rewritable while still empty). Once it has moved, the divergence is LOGGED naming both documents and the entry is left alone — unwinding a posted entry is a correcting entry's job (`reverses:`), not a silent overwrite. The comparison is order-insensitive (row order is not a query guarantee) over the union of every cell the item rows assign (`itemComparedProps`), and numbers compare by VALUE so a rescaled amount is not a change; `amendableGuard`/`itemComparedProps` are pre-rendered into the glue like everything else, and both default to the pre-#7071 behaviour when a `.glue` predates them. Concurrent-redelivery de-duplication is best-effort (a check-then-act on the back-reference) until a real UNIQUE key on the back-reference lands with schema constraint emission. Storno/negation mode LANDED as **`reverses:`** (paired with the `transitions:` void primitive - the "void-document event" is a transition into the void status): a reversal posting inherits creates/backReference/rule/map/items from the reversed sibling, negates every item amount expression on the SAME side (`Calc.eval("-()", ...)` - red storno), locates the original through the empty `storno:` self-link (none -> fail-soft skip), stamps the link on its creation, and both handlers' idempotency guards discriminate by that link (reversal counts linked rows, the sibling counts unlinked ones - `stornoProperty`/`stornoFilterProperty` in the glue). The explicit manual Reverse action (no source void) remains a follow-up. Compensation, not a transaction, is how a bad post is unwound. - **Lifecycle-aware aggregates: seed-row `stage:` + report `scope:` + symbolic status names (#6645).** An aggregate over an entity carrying a `function: EntityStatus` was **wrong by default** - drafts nobody had issued, cancelled and voided (анулиране) rows all landed in the sum unless the author remembered a magic-number status predicate in `filter:`, and nothing said so (the motivating case: a voided invoice kept its 2000 in "Revenue this month" because the report declared dimensions + measures and no `filter`, so the emitted query had no `WHERE` at all). Four coordinated pieces, all in `LifecycleStages` + `ReportIntentGenerator.scopePredicate` + `StatusSymbolResolver`: (1) a status **seed row** classifies what the status MEANS with a closed-vocabulary `stage: draft|live|cancelled|void` - metadata, never a column (the CSV generator only emits declared fields + referenced FKs, and `CsvimIntentGeneratorTest` pins that); (2) a report declares `scope: all` or a stage name, emitted as `."" IN ()` ANDed onto the filter; (3) with the nomenclature classified, an **aggregating** report **defaults to `live`** - but only when its dimensions/`filter` do not already reference the status (a breakdown BY status must keep its draft rows, and an authored predicate is authoritative), so an existing model is byte-identical until it adopts `stage:`; (4) every site that names a status accepts the **seeded name** (`from: [ISSUED]`, `setStatus: VOIDED`, `init: DRAFT`, `setRelationField` `value:`, `abortOn.status`, a check's `status`/`setStatus`, `immutableWhen`, a posting's `event.when`, a report's `filter`) - resolved on the **raw YAML tree before the typed Gson mapping** (the `rejectRemovedNumberKeys` precedent), so every validator, generator and template keeps seeing plain integers. **Why names matter more than they look:** an id is positional, so inserting a status mid-nomenclature shifts every later id and silently retargets every guard authored against the old numbering - that is how a `reverses:` posting guarded `when: "Status == 8"` stopped matching a Void that now writes 9, leaving the ledger with a receivable for a document that no longer existed, with well-formed Java emitted throughout. **Boundaries, deliberate:** the nomenclature must be seeded IN THIS MODEL - the parser holds one file and no repository, so a **cross-model** status can neither be stage-scoped nor named (both fail loudly naming the numeric-id fallback; cross-model symbols need the name→id map on the generated `.model` and are follow-up work). A symbolic **ordering** comparison (`Status >= ISSUED`) is rejected - names have no order, that is what `scope:` is for. A nomenclature that declares its own `stage` property collides with the marker and is rejected rather than guessed. Nothing is emitted into the `.model` for `stage` - no consumer needs it yet (the Harmonia badge's `statusVariant` keyword guess is the obvious future one). **Part 3, the cheap half that catches everything the other three cannot:** when a report aggregates over a lifecycle entity and neither declares `scope:` nor filters on the status AND the nomenclature is unclassified, generation records a `context.addIssue` warning - surfaced in the generate response's `warnings` and now in the **Intent Editor**'s own amber strip (it used to discard them on success; the Builder shell already showed them). That warning, not the default, is what turns an invisible modelling omission into a visible one. - **`lifecycle:` on an entity = the declarative state machine (#6714).** The whole set of legal status edges, declared once over the entity's `function: EntityStatus` nomenclature (`edges: [{ from: DRAFT, to: [ISSUED, CANCELLED] }, ...]`, either side a seeded name or an id) and **enforced on every status write**. The gap it closes: the status machinery was a set of point constructs - `init:` names the start, a `transitions:` button guards the flips that go through THAT button, a workflow `setRelationField` writes one unguarded, a `checks:` rejection files another - and nothing declared which edges were legal at all, so any other writer (a workflow branch, a glue action, a plain REST call) could jump a document from any status to any other and nothing noticed. **Enforcement lives in the generated REPOSITORY, deliberately** (`Repository.java.template`: `LIFECYCLE_EDGES` + `enforceLifecycle` / `enforceLifecycleMove` / `enforceLifecycleStart`, `ValidationException` -> 400) - it is the ONE choke point every writer passes through: `update` (the REST payload), `updateWithoutEvent` (system writes), and `updateProperties` (which `updateProperty`, and therefore the transition controller, the workflow setters and `updateDerived`, all route through - so the targeted-write overrides are now emitted for a lifecycle entity too, not only for `documentChecks`/`hasLabel`). Guarding the transition endpoints instead would have left every other writer free, which is the whole defect. `enforceLifecycleStart` (emitted only when the status relation declares `init:`) additionally refuses a CREATE filed anywhere but at the start - entering the lifecycle mid-graph skips it rather than travelling it - and is placed BEFORE the aggregate-guard macros in `save()` so an `outcome: reject` can still file the record where the model says. Emission is three scalars on the entity map (`lifecycleStatusProperty`, `lifecycleEdges` as `1>2,1>9` pairs, `lifecycleStatusNames` as `1=DRAFT,...` so a rejection reads "cannot move from ISSUED to DRAFT" instead of quoting positional ids, plus `lifecycleInitialStatus`) - scalars, so they reach the `.edm` twin like `immutableStatusValues`. **Parse-time is where the other status sites are made to agree** (`validateLifecycles`): every `from` of a `transitions:` entry must reach its `setStatus` along an edge (a button is presentation over the graph), and a status written by a `setRelationField` step or forced by a check's rejection must be one some edge reaches - which is what catches a reject path transiting through an approved status when the file is read. **Deliberate boundaries:** no `on:` key - the graph is always over the EntityStatus relation, so naming it would be redundant, and YAML 1.1 reads a bare `on` as the boolean `true` (it would arrive as the key `true` and bind to nothing), so `rejectLifecycleOn` refuses it in the raw-tree preprocessing rather than dropping it silently; a cross-model nomenclature is seeded in its owner model and so is its lifecycle (refused, naming that); the nomenclature must be seeded here (the ids are validated against the seeds); no reachability check - one nomenclature may serve two entities with different graphs, so "unreachable here" is not an error. - **`immutableWhen:` / `immutable:` on an entity = user-write immutability.** `immutableWhen: "Status == 2"` (a boolean expression over EntityStatus seed ids, terms joined with `||`) makes update/delete through the generated REST controller answer 409 CONFLICT while the record's `function: EntityStatus` FK satisfies it; `immutable: true` is the unconditional append-only variant (mutually exclusive with `immutableWhen`; a non-existent id still yields 404, not 409). Emitted as the entity-level `immutableStatusProperty` + `immutableStatusValues` (or `immutableAlways`) model attrs; `requireMutable` fetches the existing row before writing. Repository writes are deliberately unaffected — the workflow (storno generation, roll-ups, ProcessId write-back) keeps working; this guards the USER surface, per the accounting audit-trail requirement (corrections are reversals, never edits). **The UI is gated up front, not just on the 409:** each of the three generated controllers (power / partner / my) also exposes a **`GET /{id}/mutable`** pre-check (`{"mutable": true|false}` via the shared `isMutable`, scoped like its reads), and every Harmonia surface consumes it — the manage form and document pages ask it on edit load and force the read-only preview mode with a "Read-only" title badge (so a directly typed `/edit` URL opens read-only), the partner/my form + document pages disable their controls (`fieldset :disabled`) and hide Save/Delete/item actions, while the browse tables (manage list, master) gate row Edit/Delete through a **baked `isRowImmutable(row)`** computed from the row's status FK against the generation-time immutable ids — no per-row API call, same generated-from-the-same-attrs no-drift argument as the client `validationSchema`. The pre-check fails OPEN (an outage must not lock the UI); the PUT/DELETE 409 stays the authoritative guard. Covered by `IntentEmissionCoverageIT` (endpoint tokens + page tokens + mutable=false/true over REST). Parser requires an EntityStatus relation. Alongside it (no DSL): every generated controller now maps a **database constraint violation on DELETE to 409** ("referenced by other records") instead of a 500. Scope of that mapping: the schema template does emit `type: "foreignKey"` structures, but `SchemasSynchronizer.parseImpl` drops them **by design** — a foreign key never becomes a database constraint on this platform, because a constraint binds insert/delete ORDER into the schema where seeds, imports, regeneration and deletes would all have to obey an ordering nothing in the model asked for; referential integrity is a business-layer check. Only the **unique** keys are carried over (`carryUniqueConstraints`, #6793), so the 409 engages for a business-key collision and never for a reference. Anything that must not outlive the record it points at therefore needs an explicit handler — which is what an expansion's `OnDelete` cleanup is (#6821). Date-based period locking (records whose date falls in a Locked period) is deliberately NOT part of this — its shape needs the real fiscal-period module and follows as its own PR. **The lock reaches the master's composition CHILDREN (#6695).** It was per-entity, and a child declares no immutability of its own — while its generated repository writes THROUGH to the master, recomputing `net`/`vat`/`total` on every `save`/`update`/`delete`. So `POST`/`PUT`/`DELETE` on a line of an ISSUED invoice succeeded over REST and silently rewrote the document's totals after the number was stamped, the immutable snapshot taken and the ledger posted — the UI forbade it, REST permitted it, and the permitted operation was the one `immutableWhen` exists to prevent. `ModelParameterProcessor.inheritMasterLock` now propagates the master's `immutableAlways` / `immutableStatusProperty` + values onto each direct composition child as a `masterLock` map (master entity + FK property + its `…Entity`/`…Repository` classes, resolved through the composition FK's perspective exactly as the personal/partner inheritance does), and all three generated controllers (power / partner / my) emit a `requireMasterMutable` that loads the master and answers the same 409 — on create (the payload's FK), on update (the STORED master *and* the incoming one, so a line cannot be moved into a locked document either), on delete, and on an attachment upload. Engine writers stay exempt by construction: they go through the repository, not the controller — which is why the issue-time snapshot generator (`Attachments.store` + `repository.save`) is untouched. The opt-out is the flag #6700 already introduced: `locksWithMaster: false` on the child (settlement is a different lifecycle from content), so the affordance and the REST guard are governed by one declaration and cannot drift apart. Only the DIRECT child is covered — that is the shape that writes through to the master. It composes with the prompted `generates` action (#6685): that create runs through the TARGET's repository, not a controller, so a guided create against a post-issue child keeps working on a locked document exactly as its per-record button (deliberately not gated on mutability) implies — the panel and the action remain the two separate answers to "this collection must go on being recorded". `IntentEmissionCoverageIT` carries both controls: `EntryLine` (silent → inherits) is refused create/update/delete on a POSTED entry and the master's total is asserted UNMOVED, while `CampaignNote` (`locksWithMaster: false`) still posts to a locked campaign. diff --git a/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template b/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template index f49c8b56ad7..9d19e81d40c 100644 --- a/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template +++ b/components/template/template-application-events-java/src/main/resources/META-INF/dirigible/template-application-events-java/events/Posting.java.template @@ -30,7 +30,9 @@ import org.eclipse.dirigible.sdk.utils.Json; * genuinely failed post is undone by a compensation action, not this handler; * - a missing rule row or a null referenced rule column SKIPS the posting (the document stays on the * unposted worklist - final-status documents with no back-referencing ${targetEntity}); - * - all writes go through the generated repositories (numbering, status init, validations, checks); + * - all writes go through the generated repositories (numbering, status init, validations, checks), + * and the whole post - the stale rows a rewrite replaces, the header, every derived line - is ONE + * transaction, so a refused line leaves the previous post standing rather than a partial one; * - REVERSAL mode (intent reverses:): the handler locates the ORIGINAL document (back-reference set, * storno link empty), skips fail-soft when none exists, and creates the negated copy (same sides, * negative amounts - red storno) with the storno link stamped; its idempotency guard counts only @@ -92,9 +94,9 @@ public class ${className}Posting implements MessageHandler { } #end #end - // Idempotent + resumable + amendable posting (cloud-native: no cross-step transaction). The - // back-reference identifies an existing post; comparing it with the freshly derived content is - // what tells the three cases apart: identical -> a redelivery, nothing to do; different -> + // Idempotent + resumable + amendable posting. The back-reference identifies an existing post; + // comparing it with the freshly derived content is what tells the three cases apart: + // identical -> a redelivery, nothing to do; different -> // either a HALF-post to complete or an AMENDED source to rewrite the post from (#7071). // Concurrent-redelivery de-duplication is best-effort until a UNIQUE key on the back-reference // lands with schema constraint emission; a genuinely failed post is reversed by a compensation @@ -160,10 +162,12 @@ public class ${className}Posting implements MessageHandler { #end gen.${javaGenFolderName}.data.${targetJavaPerspective}.${targetEntity}Entity target; boolean refresh = !existingTargets.isEmpty(); + // The rows the post carries today: the ones a rewrite replaces, empty for a fresh post. Both + // branches assign it exactly once so the write block below can close over it. + java.util.List currentItems; if (refresh) { target = existingTargets.get(0); - java.util.List currentItems = - itemsRepository.findAll(Criteria.create().eq("${itemsFk}", target.${targetPk})); + currentItems = itemsRepository.findAll(Criteria.create().eq("${itemsFk}", target.${targetPk})); boolean unchanged = currentItems.size() == derivedItems.size(); #foreach($a in $headerAssignments) unchanged = unchanged && same(target.${a.targetProp}, ${a.expr}); @@ -197,28 +201,39 @@ public class ${className}Posting implements MessageHandler { return; } #end - // Either a HALF-post (an item write failed after the target was saved) or an AMENDMENT - // (the source was re-issued with different content): both converge on rewriting the post - // to what the source derives now. - for (gen.${javaGenFolderName}.data.${itemsJavaPerspective}.${itemsEntity}Entity stale : currentItems) { - itemsRepository.delete(stale); - } } else { target = new gen.${javaGenFolderName}.data.${targetJavaPerspective}.${targetEntity}Entity(); target.${backRefProperty} = source.${sourceKeyField}; #if($stornoProperty != "") target.${stornoProperty} = original.${targetPk}; #end + currentItems = java.util.List.of(); } #foreach($a in $headerAssignments) target.${a.targetProp} = ${a.expr}; #end - gen.${javaGenFolderName}.data.${targetJavaPerspective}.${targetEntity}Entity saved = - refresh ? targetRepository.update(target) : targetRepository.save(target); - for (gen.${javaGenFolderName}.data.${itemsJavaPerspective}.${itemsEntity}Entity item : derivedItems) { - item.${itemsFk} = saved.${targetPk}; - itemsRepository.save(item); - } + // ONE transaction for the whole post: the stale rows a rewrite replaces, the header and every + // derived line either all become durable or none of them does. Written as separate + // transactions, a line the item repository refuses (a validation, a constraint) left the + // journal entry with a header and a PARTIAL set of lines - and unlike the half-post case there + // is no second event to self-heal from, so the ledger ended up worse than the stale but + // balanced state the rewrite set out to fix (dirigible #7132). A rewrite therefore deletes the + // rows it replaces inside the same unit: either the post says what the source derives now, or + // it still says what it said before. + org.eclipse.dirigible.components.data.store.java.repository.UnitOfWork.run(() -> { + // Either a HALF-post (an item write failed after the target was saved) or an AMENDMENT + // (the source was re-issued with different content): both converge on rewriting the post + // to what the source derives now. Empty for a fresh post, so no branch is needed. + for (gen.${javaGenFolderName}.data.${itemsJavaPerspective}.${itemsEntity}Entity stale : currentItems) { + itemsRepository.delete(stale); + } + gen.${javaGenFolderName}.data.${targetJavaPerspective}.${targetEntity}Entity saved = + refresh ? targetRepository.update(target) : targetRepository.save(target); + for (gen.${javaGenFolderName}.data.${itemsJavaPerspective}.${itemsEntity}Entity item : derivedItems) { + item.${itemsFk} = saved.${targetPk}; + itemsRepository.save(item); + } + }); } /** diff --git a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEmissionCoverageIT.java b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEmissionCoverageIT.java index b7e6f121bae..6a8f26c5025 100644 --- a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEmissionCoverageIT.java +++ b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEmissionCoverageIT.java @@ -3024,6 +3024,19 @@ private void assertEmission() { "an existing post must be compared cell by cell against what the source derives now"); assertTrue(basePosting.contains("targetRepository.update(target) : targetRepository.save(target)"), "a diverging post must be rewritten in place, never doubled"); + // #7132: and every write of that rewrite is ONE transaction. A refused line (a validation, a + // constraint) must leave the previous post standing - a header with a partial line set has no + // second event to self-heal from, so it is worse than the stale but balanced post it replaced. + // Asserted by POSITION: the delete of the stale rows and the header write must both sit inside + // the block, which a mere "the file mentions UnitOfWork" check would not tell apart. + int unitOfWork = basePosting.indexOf("UnitOfWork.run(() -> {"); + assertTrue(unitOfWork > 0, "the posting's write phase must run in a UnitOfWork"); + assertTrue(basePosting.indexOf("itemsRepository.delete(stale)") > unitOfWork, + "the stale rows a rewrite replaces must be deleted inside the unit of work, not before it"); + assertTrue(basePosting.indexOf("targetRepository.update(target)") > unitOfWork, + "the header write must run inside the unit of work"); + assertTrue(basePosting.indexOf("itemsRepository.save(item)") > unitOfWork, + "the derived lines must be written inside the unit of work"); assertTrue(basePosting.contains("-Doc-transitioned"), "a status-triggered posting must bind the -transitioned topic"); // source-FK copy (#6533): a to-one relation item cell copies the source FK verbatim onto the // line - no Calc, no negation, and it must carry through UNCHANGED onto the reversal line. diff --git a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java index 1a8264e60ac..a021dc20c61 100644 --- a/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java +++ b/tests/tests-integrations/src/main/java/org/eclipse/dirigible/integration/tests/api/IntentEngineIT.java @@ -3126,10 +3126,11 @@ void postings_generates_the_idempotent_resumable_handler() { assertTrue(glue.contains("\"postings\""), "the .glue should carry the postings collection"); assertTrue(glue.contains("OrderLedger"), "the posting className should be carried in the glue"); - // Events template: the generated handler is idempotent + resumable + amendable (the - // cloud-native posting semantics - no cross-step transaction). It derives the full content - // first and compares it with the existing post: identical is a no-op, different is either a - // half-post to complete or an amended source to rewrite from (#7071). + // Events template: the generated handler is idempotent + resumable + amendable. It derives the + // full content first and compares it with the existing post: identical is a no-op, different is + // either a half-post to complete or an amended source to rewrite from (#7071). The writes that + // rewrite are ONE transaction (#7132) - across STEPS the model stays non-transactional, a bad + // post being unwound by a correcting entry. generateFromModel("template-application-events-java/template/template.js", "postingtest.glue"); String posting = codeOf("gen/events/postingtest/OrderLedgerPosting.java"); assertTrue(posting.contains("implements MessageHandler"), "the posting is a self-describing message handler"); @@ -3141,6 +3142,14 @@ void postings_generates_the_idempotent_resumable_handler() { assertTrue(posting.contains("itemsRepository.delete(stale)"), "a stale or partial item set is cleared before the rewrite"); assertTrue(posting.contains("targetRepository.update(target) : targetRepository.save(target)"), "an existing post is rewritten in place, a fresh one created"); + // #7132: and all of it in one transaction - asserted by POSITION, since a header written outside + // the block with the lines inside it would still "mention UnitOfWork". + int unitOfWork = posting.indexOf("UnitOfWork.run(() -> {"); + assertTrue(unitOfWork > 0, "the write phase runs in a unit of work"); + assertTrue(posting.indexOf("itemsRepository.delete(stale)") > unitOfWork, + "the stale rows are deleted inside the unit of work, not before it"); + assertTrue(posting.indexOf("targetRepository.update(target)") > unitOfWork, "the header is written inside the unit of work"); + assertTrue(posting.indexOf("itemsRepository.save(item)") > unitOfWork, "the derived lines are written inside the unit of work"); // The Ledger carries no status lifecycle, so there is nothing to act on and nothing to guard. assertFalse(posting.contains("was NOT rewritten"), "a target with no status lifecycle is always rewritable"); }