intent: a create-from's items: has a source-row rule - where: + refuse: (#7091)

.claude/docs/intent-layer.md

@@ -32,6 +32,8 @@ A single `app.intent` YAML file at a project root is the source of truth one alt
 
 **The enrichment channel (`phases:` + `onPhase`, [#6929](https://github.com/eclipse-dirigible/dirigible/issues/6929)):** a value a listener computes AFTER the insert — a moving-average cost, a snapshot column, an external lookup — must be written back **event-silently** or it re-fires every onUpdate consumer of a change the user never made; so it published nothing at all, and a declarative consumer of that value had no moment to bind. Bound to `onCreate` it RACED the enrichment (two listeners on one topic have no order — each `MessageHandler` is its own durable subscriber, and there is no priority anywhere), and posted a balanced-looking journal entry for a null amount with parse, generation, compile and publish all green. The fix is a CHANNEL, not an ordering contract the broker cannot keep: an entity declares the moments it announces (`phases: [costed]`), the Java DAO template emits one **`announce<Phase>(id, values)`** per phase — `updateProperties` with the phase's own topic, so the enrichment and its notice ride ONE write into the outbox and commit together — and any glue consumer binds `event: { onPhase: <Entity>, phase: <name> }`. The generated method is the point: a hand-typed topic string reproduces exactly the silence being removed, a mistyped `announceCosted` is a compile error. Accepted by `postings:` (the driver), `notifications:`, `integrations:`, `outbound:` and an event-driven `generates:`, with the `when:` guard optional there (the phase already IS one moment); deliberately not by a process `trigger:`, a `wait` or `resolves:`. Refused at parse, each because it is otherwise silent: a phase that is not a lower-camel identifier, one named after a platform channel (`updated`/`deleted`/`transitioned`/`rekeyed`), a duplicate, a `phase:` key on another axis, and a binding naming a phase the entity does not declare. Details in the engine-intent guide's phases bullet.
 
+**Which source rows become lines (`items: where:` + `refuse:`, [#7091](https://github.com/eclipse-dirigible/dirigible/issues/7091)):** a create-from's mirror `items:` block cloned EVERY row of the source document into a target line and the DSL could not say which rows qualified, so base-timesheets billed every member timesheet of the project-month - a DRAFT / REJECTED one at the same footing as an APPROVED one, and an EMPTY one (whose mapped quantity the target refuses) stopped the whole Generate until someone deleted the row by hand. "Invoice the approved month" is the one flow a billing clerk runs, and the module could either bill unapproved hours or not bill at all; the gap is fleet-wide (proforma -> invoice, quotation -> order, order -> invoice). `where:` is the rule - the same `{ field, op, value }` triples a `schedules[].where` carries, incl. a moment value resolved against the clock of the run - pushed into the very `Criteria` that already selects the source's rows by their master foreign key, so an unqualified row is never loaded; a condition naming the source ITEM's own `function: EntityStatus` relation may use the seeded status name (on the item's nomenclature, never the header's). `refuse:` declares the other reading: an unqualified row stops the whole run with the authored message plus the KEYS of the offending rows, instead of being left out - dropping a rejected line silently and billing it silently are both wrong for different months, so skipping is the default and `refuse:` without a `where:` is refused at parse. **A rule that qualifies no row refuses too**, rather than committing a header with no lines - the harder failure to notice, the document existing and counting as the period's billing. Scoped to a `where`-declaring block, so a rule-less items block is byte-identical; the rule's `field` is checked against the item source's own properties at parse, unlike a schedule's query, whose source may be a cross-model row.
+
 **A create-from is not offered twice (`fromStatus:`, [#7068](https://github.com/eclipse-dirigible/dirigible/issues/7068)):** a `generates:` with a `sourceStatus:` completion hook flipped its source once the target existed and then went on offering the same button on the flipped record - and answering the same endpoint 200 - so a second click minted a **second document**: a proforma already INVOICED produced a second invoice, in the customer's hands. The hook declared what "already done" looks like; nothing consulted it. A create-from now carries a from-status guard resolved ONCE and fed to both halves of the action: the generated `run()` refuses with **409** before anything is created, and the contributed action descriptor carries the same guard so the shared `customActions` store stops OFFERING the click on a record it would refuse (`getActions(view, type, record)` takes the record the view already has). Two shapes: `fromStatus: [...]` is the explicit allow-list - the `from:` of a `transitions:` entry, spelled differently only because `from:` on a create-from already names the source ENTITY - and absent it a declared `sourceStatus` IMPLIES the deny-list of exactly that status, so a model that already carries the defect is fixed with no authoring change. The guard is on the CLICK: an event-driven create-from keeps its own at-most-once back-reference guard and qualifies its moment with `event.when`, so `fromStatus` on an event-only rule is refused at parse rather than silently ignored - as are a `page` scope, a source with no `function: EntityStatus` relation, and an allow-list containing the `sourceStatus` the action itself writes.
 
 **An amended source rewrites its posting ([#7071](https://github.com/eclipse-dirigible/dirigible/issues/7071)):** the amend path (Confirm → Reject → edit the lines → Issue again) raises a `postings:` trigger a SECOND time, and the old idempotency test - an existing post whose item count reached the derived one - read that as "already posted", so the journal entry silently kept the amounts of the previous issue while the invoice it references had moved on. No second entry (right), a ledger short by the difference (wrong), and nothing anywhere said so. The generated handler now derives the WHOLE content first and compares it with what the post carries: identical is a redelivery (no-op), different is either a half-post to complete or an amendment to REWRITE the post from - header assignments re-applied, items replaced, never a second document. The rewrite stops where the created document's own lifecycle says someone has taken it over: it is rewritable only while its `function: EntityStatus` relation still holds the `init:` the posting's own create wrote (with no status lifecycle there is nothing to act on, so it is always rewritable), and past that the divergence is logged naming both documents and left to a correcting entry - `reverses:` - rather than overwritten behind the accountant's back. The comparison is order-insensitive over every cell the item rows assign, and numbers compare by value so a rescaled amount is not a change.

components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/GlueIntentGenerator.java

@@ -1096,6 +1096,12 @@ private static List<Map<String, Object>> buildGenerates(IntentModel model, Map<S
                                 temporalKinds(crossModel ? null : byName.get(items.getTo()), itemTarget),
                                 relationProperties(crossModel ? null : byName.get(items.getTo()), itemTarget)));
                 e.put("itemLines", new ArrayList<>());
+                // The source-row rule (issue #7091), pre-rendered as the tail of the Criteria that
+                // already selects the source's item rows by their master foreign key - so the rows the
+                // rule excludes are never loaded, and a rule of no conditions renders the empty string
+                // and therefore the query this always ran.
+                e.put("itemWhere", ScheduleSupport.conditionChain(items.getWhere()));
+                e.put("itemRefuse", items.hasWhere() && items.hasRefuse() ? items.getRefuse() : "");
             } else if (hasItemLines) {
                 // The synthetic lines write into the TARGET document's composition line-items child,
                 // resolved automatically (never named in the intent): same-model from this model,
@@ -1132,6 +1138,8 @@ private static List<Map<String, Object>> buildGenerates(IntentModel model, Map<S
                 e.put("srcFkProperty", "");
                 e.put("toFkProperty", IntentNaming.pascalCase(g.getTo()));
                 e.put("itemFieldAssignments", new ArrayList<>());
+                e.put("itemWhere", "");
+                e.put("itemRefuse", "");
                 // Cell expressions are written over the SOURCE record, so the known-property set comes
                 // from wherever the source is defined - locally, or the owner .model for a cross-model
                 // source (an unresolved owner yields an empty set, i.e. no local name check).
@@ -1148,6 +1156,8 @@ private static List<Map<String, Object>> buildGenerates(IntentModel model, Map<S
                 e.put("toFkProperty", "");
                 e.put("itemFieldAssignments", new ArrayList<>());
                 e.put("itemLines", new ArrayList<>());
+                e.put("itemWhere", "");
+                e.put("itemRefuse", "");
             }
             e.put("hasPrompt", g.hasPrompt());
             e.put("promptFields", promptFields(g, crossModel ? null : byName.get(g.getTo())));

components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/generator/ScheduleSupport.java

@@ -12,6 +12,7 @@
 import java.time.Duration;
 import java.time.Period;
 import java.time.format.DateTimeParseException;
+import java.util.List;
 import java.util.Map;
 import java.util.regex.Matcher;
 import java.util.regex.Pattern;
@@ -175,8 +176,24 @@ public static boolean isSupportedOperator(String op) {
      *         {@code Criteria.create().lt("DueOn", java.time.LocalDate.now()).eq("Status", "ACTIVE")}
      */
     public static String criteriaExpression(ScheduleIntent schedule) {
-        StringBuilder expr = new StringBuilder("Criteria.create()");
-        for (ScheduleConditionIntent condition : schedule.getWhere()) {
+        return "Criteria.create()" + conditionChain(schedule.getWhere());
+    }
+
+    /**
+     * The same conditions as a chain of {@code Criteria} calls with no {@code Criteria.create()} in
+     * front, so a caller that has already opened a criteria can append them - a create-from's
+     * source-row rule ({@code items: where:}, issue #7091), which narrows the very query that selects
+     * the source document's item rows by their master foreign key.
+     *
+     * @param conditions the authored conditions, may be {@code null}
+     * @return e.g. {@code .eq("Status", 3).gt("TotalHours", 0)}, or the empty string for no conditions
+     */
+    public static String conditionChain(List<ScheduleConditionIntent> conditions) {
+        if (conditions == null) {
+            return "";
+        }
+        StringBuilder expr = new StringBuilder();
+        for (ScheduleConditionIntent condition : conditions) {
             String method = OPERATORS.get(condition.getOp());
             if (method == null) {
                 continue; // validated at parse time; defensively skip an unknown operator

components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/model/GeneratesItemsIntent.java

@@ -10,6 +10,7 @@
 package org.eclipse.dirigible.components.intent.model;
 
 import java.util.LinkedHashMap;
+import java.util.List;
 import java.util.Map;
 
 /**
@@ -19,6 +20,14 @@
  * {@link #defaults} follow the same semantics as on the parent (source copy vs
  * {@code now}/literal). The foreign key back to the master is set automatically - it must not be
  * listed in {@link #map}.
+ *
+ * <p>
+ * {@link #where} is the SOURCE-ROW RULE (issue #7091): without it every row of the source document
+ * became a line, so a document could only be generated whole - an unapproved timesheet was billed
+ * at the same footing as an approved one, and an empty one (whose mapped value the target refuses)
+ * stopped the whole create-from until someone deleted the row by hand. {@link #refuse} declares
+ * which of the two readings the document means: dropping an unqualified row silently and billing it
+ * silently are both wrong, for different months.
  */
 public class GeneratesItemsIntent {
 
@@ -34,6 +43,37 @@ public class GeneratesItemsIntent {
     /** Target item property -> {@code now} or a literal value. */
     private Map<String, String> defaults = new LinkedHashMap<>();
 
+    /**
+     * Optional source-row rule (issue #7091): only the rows of {@link #from} that satisfy every
+     * condition become target lines. The conditions are the same field/op/value triples a
+     * {@code schedules[].where} carries and are pushed into the very {@code Criteria} that already
+     * selects the source's rows by their master foreign key, so the unqualified rows are never loaded.
+     *
+     * <p>
+     * A condition naming the source item's {@code function: EntityStatus} relation may use the seeded
+     * status NAME - an id is positional, and a status inserted mid-nomenclature would otherwise
+     * silently retarget the rule.
+     *
+     * <p>
+     * With a rule declared, a source whose rows ALL fail it refuses the create-from rather than
+     * committing a header with no lines: a document of no lines is not the document that was asked for,
+     * and the empty invoice is the harder failure to notice of the two.
+     */
+    private List<ScheduleConditionIntent> where;
+
+    /**
+     * Optional refusal message (issue #7091): with it, a source row that does not satisfy
+     * {@link #where} stops the whole create-from - a {@code ValidationException} carrying this text and
+     * the keys of the offending rows - instead of being left out of the document.
+     *
+     * <p>
+     * Which of the two an unqualified row deserves is a property of the document, not of the platform:
+     * a rejected timesheet quietly dropped from an invoice and a rejected timesheet quietly billed are
+     * both wrong, so skipping is the default and this declares the other reading. Requires
+     * {@link #where} - there is nothing for a row to be unqualified against without it.
+     */
+    private String refuse;
+
     public String getFrom() {
         return from;
     }
@@ -65,4 +105,30 @@ public Map<String, String> getDefaults() {
     public void setDefaults(Map<String, String> defaults) {
         this.defaults = defaults == null ? new LinkedHashMap<>() : defaults;
     }
+
+    public List<ScheduleConditionIntent> getWhere() {
+        return where;
+    }
+
+    public void setWhere(List<ScheduleConditionIntent> where) {
+        this.where = where;
+    }
+
+    /** Whether a source-row rule is declared (see {@link #where}). */
+    public boolean hasWhere() {
+        return where != null && !where.isEmpty();
+    }
+
+    public String getRefuse() {
+        return refuse;
+    }
+
+    public void setRefuse(String refuse) {
+        this.refuse = refuse;
+    }
+
+    /** Whether an unqualified source row refuses the whole create-from (see {@link #refuse}). */
+    public boolean hasRefuse() {
+        return refuse != null && !refuse.isBlank();
+    }
 }

components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/model/ScheduleConditionIntent.java

@@ -10,10 +10,12 @@
 package org.eclipse.dirigible.components.intent.model;
 
 /**
- * One condition of a {@link ScheduleIntent}'s {@code where} filter: a field, a comparison operator
+ * One condition of a {@code where} filter - a {@link ScheduleIntent}'s row query, or the source-row
+ * rule of a {@link GeneratesItemsIntent} (issue #7091): a field, a comparison operator
  * ({@code eq}/{@code ne}/{@code gt}/{@code ge}/{@code lt}/{@code le}/{@code like}) and a value. The
- * value is a literal, or the token {@code CURRENT_DATE} / {@code CURRENT_TIMESTAMP} which the
- * generated job evaluates to "now". Maps to a typed {@code Criteria} condition.
+ * value is a literal, or the token {@code CURRENT_DATE} / {@code CURRENT_TIMESTAMP}, which the
+ * generated code evaluates against the clock of the run that fires. Maps to a typed
+ * {@code Criteria} condition.
  */
 public class ScheduleConditionIntent {
 

components/engine/engine-intent/src/main/java/org/eclipse/dirigible/components/intent/parser/IntentParser.java

@@ -7330,6 +7330,7 @@ private static void validateGenerates(IntentModel model, Set<String> entityNames
                 // implies a cross-model item - resolved in the owner's .model, not here.
                 validateMapTarget(crossModel || items.getTo() == null ? null : byName.get(items.getTo()), items.getMap(),
                         "generates [" + name + "]", "items map", issues);
+                validateGeneratesItemsWhere(items, "generates [" + name + "]", itemSource, issues);
             }
             if (g.hasUnique()) {
                 // The natural key is a SCHEDULE's idempotency guard (issue #7070). An on-demand
@@ -7347,6 +7348,50 @@ private static void validateGenerates(IntentModel model, Set<String> entityNames
         }
     }
 
+    /**
+     * Validate the source-row rule of a create-from's mirror items block (issue #7091): which rows of
+     * the source document become lines of the target, and what an unqualified one costs.
+     *
+     * <p>
+     * The conditions are the field/op/value triples a {@code schedules[].where} carries, checked the
+     * same way - a supported operator, and a moment value ({@code CURRENT_DATE} and friends) whose
+     * shape the compared field can carry. What is checked additionally is the {@code field} itself:
+     * unlike a schedule's query, whose source may be a cross-model row or an {@code audit:} column this
+     * model cannot see, an items rule reads a LOCAL row being cloned, so a name it does not declare
+     * could only ever be a condition the database rejects on the first click.
+     *
+     * <p>
+     * {@code refuse:} requires the rule: without conditions no row is ever unqualified, so the message
+     * is a promise nothing can keep - the class of authored-but-unconsumed key this module refuses
+     * everywhere else.
+     */
+    private static void validateGeneratesItemsWhere(GeneratesItemsIntent items, String subject, EntityIntent itemSource,
+            List<String> issues) {
+        if (items.hasRefuse() && !items.hasWhere()) {
+            issues.add(subject + " items declares refuse with no where - nothing can be unqualified without a source-row rule;"
+                    + " add the where conditions the refused rows fail");
+        }
+        if (!items.hasWhere()) {
+            return;
+        }
+        for (ScheduleConditionIntent condition : items.getWhere()) {
+            if (condition.getField() == null || condition.getField()
+                                                         .isBlank()) {
+                issues.add(subject + " items has a where-condition with no field");
+                continue;
+            }
+            if (!SCHEDULE_OPERATORS.contains(condition.getOp())) {
+                issues.add(subject + " items where-condition uses unsupported operator [" + condition.getOp()
+                        + "] (supported: eq/ne/gt/ge/lt/le/like)");
+            }
+            if (itemSource != null && !hasPropertyIgnoreCase(itemSource, condition.getField())) {
+                issues.add(subject + " items where-condition reads [" + condition.getField()
+                        + "], which is not a field or to-one relation of [" + itemSource.getName() + "]");
+            }
+            validateScheduleMoment(condition, itemSource, subject + " items", issues);
+        }
+    }
+
     /**
      * Validate the from-status guard of a create-from (issue #7068): the statuses the SOURCE may stand
      * in for the action to run at all.