rfc: a window's extent is the producer's to decide; window.start and window.end are the earliest and latest event time of the lines it contains

CHANGELOG.md

@@ -34,8 +34,10 @@ The spec follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 ## [0.10.0] — UNRELEASED
 
 **Breaking** under [`GOVERNANCE.md`](GOVERNANCE.md) §2 — a new **required** member
-on `MetaLogDiff`, a rewritten §13.2 clause, and a `diff_version` rule (§13.1.1) that
-a producer stamping an older value now violates. It also adds one optional member
+on `MetaLogDiff`, a rewritten §13.2 clause, a `diff_version` rule (§13.1.1) that
+a producer stamping an older value now violates, and a definition of `window.start`
+and `window.end` (§2.2) that a producer writing its own bounds there now violates.
+It also adds one optional member
 (`withheld_signals`, §13.2.2), which is additive on its own. MINOR bump: MAJOR stays `0`, so
 §6's *"the MAJOR field of `metalog_version` must equal the MAJOR of the spec"* is
 satisfied unchanged and **both schema files keep their `v0` filenames** — a reader
@@ -53,7 +55,10 @@ no implementer to break. Folding them here rather than minting 0.11.0 also keeps
 one number across `SPEC.md`, `metalog_version` and `diff_version`, which is the
 whole point of the first of them. **The size headline withdrawn under *Removed* is
 not from that RFC either**: it is an editorial change, raised by the editor after a
-measurement, and it changes no document's validity.
+measurement, and it changes no document's validity. **Nor is the §2.2 window
+definition under *Changed***: it is a breaking change of its own, proposed and argued
+in pull request [#14](https://github.com/CodeRoasted/metalog-spec/pull/14) under its own
+14-day comment window.
 
 ### Changed
 
@@ -85,6 +90,23 @@ measurement, and it changes no document's validity.
   stated rather than hidden: **§13.2 is no longer readable standalone**, and a reader
   must consult the schema to know the set.
 
+- **`window.start` and `window.end` are defined, and the specification stops assuming
+  how a producer decides a window's extent (§2.2).** The two instants had no
+  definition: a producer could write the bounds its own rule used or the times of the
+  lines it read, and a consumer could not tell which — so §12.1, which takes their
+  `min` and `max`, could compose two documents into an interval that meant neither.
+  They are now the earliest and the latest **event time** (a new glossary term) among
+  the lines the window contains, and a window with no line carries `start` equal to
+  `end`. Which lines a window contains, and how and when a producer decides it, is
+  implementation-defined; a producer **MAY** fix the extent only after reading beyond
+  it. The glossary, the §1 table, §5, §12.1, §16.6, §16.9, §16.10 and RATIONALE §R5
+  drop the wording that assumed a fixed interval closed as lines arrive: §5's
+  `previous_window_end` is the previous window's `window.end`, and §16.6's regime
+  note becomes a **MUST NOT**. §8 counts the empty-window equality among the window
+  relations the conformance validator decides. A producer that writes its own bounds
+  into `start`/`end` moves them to an extension (§7). RFC: pull request
+  [#14](https://github.com/CodeRoasted/metalog-spec/pull/14).
+
 ### Added
 
 - **§13.2.1 — `x-metalog-vacuous`, a per-property vacuity declaration.** The witness

RATIONALE.md

@@ -180,8 +180,8 @@ pressure handles the rest.
   Doesn't compose.
 
 The producer chooses what "previous window" means (typically the
-immediately preceding window of the same duration) and reports its
-boundary in `previous_window_end`.
+immediately preceding window) and names it in `previous_window_end`
+by that window's `window.end`.
 
 ---
 

SPEC.md

@@ -45,8 +45,13 @@ keywords: **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, **MAY**.
   `User <*> logged in from <*>`.
 - **TemplateID** — A stable, content-derived identifier for a
   template. See §3.2.
-- **Window** — A contiguous time interval over which a single MetaLog
-  is computed.
+- **Window** — The log lines a single MetaLog is computed over. How a
+  producer decides which lines a window contains is
+  implementation-defined (§2.2).
+- **Event time** — The time a producer attributes to a log line: the
+  timestamp the line carries, when it carries one the producer can
+  read. How a producer attributes a time to a line that carries none
+  is implementation-defined.
 - **Producer** — A program that consumes log lines and emits MetaLog
   documents.
 - **Consumer** — A program that reads MetaLog documents (dashboard,
@@ -70,7 +75,7 @@ top-level fields:
 |---|---|---|---|
 | `metalog_version` | string | yes | Spec version this document conforms to. SemVer string (e.g. `"0.10.0"`). The `MetaLogDiff` (§13) carries the same axis as `diff_version` — see §13.1.1. |
 | `producer` | object | yes | Identifies the producing implementation. See §2.1. |
-| `window` | object | yes | The time interval covered. See §2.2. |
+| `window` | object | yes | The event-time envelope of the lines the window contains. See §2.2. |
 | `source` | object | yes | What was observed (service, host, fleet). See §2.3. |
 | `canonicalization_version` | string | no | Opaque identifier for the canonicalization rules in effect. Gates `compose()`/diff comparability. See §2.4. |
 | `retention_profile` | string | no | Opaque identifier for the retention parameters (top_k, reservoir, salience weights, diversity caps). Gates `compose()`/diff comparability. See §2.4. |
@@ -166,8 +171,30 @@ that as a coupling.
 }
 ```
 
-`start` **MUST** be strictly less than or equal to `end`
-(equality is permitted for empty / heartbeat documents).
+**How a producer decides a window's extent is outside this
+specification.** Which lines a window contains, and how and when the
+producer decides it, are implementation-defined, and this
+specification assumes no producer behaviour for that decision. In
+particular it assumes neither a fixed interval, nor a decision taken
+on any single line, nor an online or causal decision: a producer
+**MAY** fix a window's extent only after reading lines beyond it, and
+**MAY** assign lines to a window retrospectively. A document that
+carries a raw re-derivation coordinate still describes exactly the
+lines whose event time falls within that coordinate's `bounds`
+(§15.3).
+
+`start` and `end` are the earliest and the latest event time among
+the lines the window contains: the window's event-time envelope. They
+say nothing about when or how the window's extent was decided, nor
+about when the document was emitted. A producer that has bounds of
+its own — the interval its rule for deciding the extent used —
+**MAY** publish them in an extension (§7).
+
+`start` **MUST** be less than or equal to `end`; the two are equal
+when every line the window contains carries the same event time, a
+window of one line included. For a window containing no line
+(`lines_observed` 0), `start` **MUST** equal `end`, and both name the
+time the producer attributes to the window.
 `duration_seconds` **MUST** equal `end - start` rounded to the
 nearest second. If a producer cannot count `lines_observed` exactly,
 it **MUST** emit its best estimate and **SHOULD** emit an
@@ -867,11 +894,12 @@ entirely rather than emit empty fields.
 
 Quantifies *how much the system's behaviour changed* since the
 previous window. Stability is a special case of §13 Diff with the
-`previous` document being the previous closed window.
+`previous` document being the window before it in the producer's
+order.
 
 ```jsonc
 {
-  "previous_window_end": "2026-04-24T10:00:00Z",  // RFC 3339, required
+  "previous_window_end": "2026-04-24T10:00:00Z",  // RFC 3339, required, the previous window's `window.end`
   "kl_divergence": 0.043,        // number ≥ 0, KL(current || previous) over template freqs
   "js_divergence": 0.021,        // number in [0, 1], symmetric Jensen-Shannon
   "new_templates": 3,            // integer, templates seen now but not in previous window
@@ -1032,10 +1060,11 @@ the file. Clause 2 is mechanically decidable in exactly one place. `window`
 (§2.2) is the only required block whose definition states relations
 **between** its members — `start` at or before `end`,
 `duration_seconds` equal to their difference rounded to the nearest
-second, and both instants in UTC. None of the three is expressible
-in JSON Schema at any draft (`format: date-time` constrains the
-grammar, never the offset, and no keyword relates two siblings),
-and all three follow from the document alone, so
+second, both instants in UTC, and `start` equal to `end` when
+`lines_observed` is 0. None of the four is expressible in JSON
+Schema at any draft (`format: date-time` constrains the grammar,
+never the offset, and no keyword relates two siblings), and all four
+follow from the document alone, so
 [`conformance/metalog_validate.py`](conformance/metalog_validate.py)
 decides them — together with the §7
 `org.metalog.lines_observed_estimated` flag, which §2.2 admits only
@@ -1269,7 +1298,7 @@ to a 1-hour MetaLog).
 
 - `C.window.start = min(A.window.start, B.window.start)`
 - `C.window.end   = max(A.window.end,   B.window.end)`
-- `C.window.duration_seconds = C.window.end - C.window.start` (real time, **not** sum of inputs)
+- `C.window.duration_seconds = C.window.end - C.window.start` (the event-time envelope of both inputs' lines, **not** the sum of their durations)
 - `C.window.lines_observed = A.window.lines_observed + B.window.lines_observed`
 - `C.source` is `A.source` if equal to `B.source`, otherwise the
   most-specific common prefix (e.g. same `fleet`, drop differing
@@ -2235,11 +2264,10 @@ This cross is a **hard firewall (normative)**:
   entry with its cube coordinate.
 - `cube_coord` is a **pure function** of the entry's `(level, where-path)` — no new
   non-determinism, no float.
-- **Regime precondition (D9):** valid **only** while the reservoir and the cube close
-  on the **same window boundary** (the fixed-window regime — true today). Under
-  adaptive window closure the two may close on different boundaries, so the cross
-  would point at a cell of the wrong window; it **MUST** be re-designed before reuse
-  there (reuse unchanged is *out-of-regime*, not a bug).
+- **Regime precondition:** valid **only** while the reservoir and the cube are
+  computed over the same lines, one window's content. A producer whose reservoir and
+  cube can cover different lines **MUST NOT** emit `cube_coord`: the cross would
+  point at a cell computed over other lines.
 
 ### 16.7 Two scales — intra-window and compose
 
@@ -2272,9 +2300,10 @@ the source of truth for any categorical marginal and does **not** reshape `stats
 
 ### 16.9 Determinism
 
-The cube is computed **in batch over the closed window** — a finite, frozen, ordered
-set — so it is a **pure function** of that set and **bit-identical across stdlibs and
-operating systems** (the standing cross-stdlib / cross-OS diagonal). Specifically:
+The cube is computed **in batch over the window's final set of lines** — a finite,
+frozen, ordered set — so it is a **pure function** of that set and **bit-identical
+across stdlibs and operating systems** (the standing cross-stdlib / cross-OS
+diagonal). Specifically:
 
 - `count` is integer; the closure and the border are set operations; cells and
   border cells **MUST** serialise in canonical (coord-sorted) order.
@@ -2284,7 +2313,7 @@ operating systems** (the standing cross-stdlib / cross-OS diagonal). Specificall
   carries **one** `canonicalization_version` bump and its golden cascade in the same
   pass.
 - The §16.10 collapse is **deterministic content**: the trigger reads only the
-  closed window (never wall-clock, never an environment budget); the policy is pure
+  window's lines (never wall-clock, never an environment budget); the policy is pure
   integer (`Δcells / cost` by cross-multiplication — no float); and the fixed
   candidate order (LEVEL before WHERE) **is** the declared total-order tie-break.
   The policy is **version-stamped** — changing its budget, costs, steps or
@@ -2304,7 +2333,7 @@ budget-driven dimensional collapse** — three separated objects:
   declaring it is what makes it knowable to the **consumer**. It dominates every
   other term of the size formula (§11.4), so an undeclared budget leaves the
   consumer unable to price the block at all.
-- **TRIGGER** — a pure function of the closed window's content: closed cells over
+- **TRIGGER** — a pure function of the window's content: closed cells over
   budget. **Closure-first, collapse-last**: when closure alone fits, nothing
   degrades — collapse is the rare guard, not the normal path.
 - **POLICY** — while over budget, apply the best **admissible** monotone coarsening

conformance/README.md

@@ -144,13 +144,13 @@ never looked.
 
 ## Why the self-test exists
 
-A validator that cannot fail is decoration. `--selftest` runs twenty-six fixtures whose
+A validator that cannot fail is decoration. `--selftest` runs thirty-one fixtures whose
 expected results are **hand-authored in `fixtures/manifest.json` from the spec and
 the schemas** — never captured from a run, because an expectation copied out of the
 tool under test makes the tool its own oracle, and the pair then agree forever
 while both are wrong.
 
-Fourteen of those fixtures carry a `control` tag, naming **twelve** distinct blindnesses
+Nineteen of those fixtures carry a `control` tag, naming **seventeen** distinct blindnesses
 (`instrument-failure` is carried by three), and the self-test **refuses to run**
 if any tag is missing from the manifest — deleting a fixture cannot quietly widen
 what a green covers. Each control forecloses one specific way this validator could
@@ -170,6 +170,11 @@ have gone green while blind:
 | `witness-rule-unchanged-arm` | `"unchanged"` forbids one, and a rule that binds only the other arm is satisfied forever by a producer that always writes `"unchanged"`. The fixture is the same control with **one row** changed — `template_deltas[0]` from `delta: 0` to `delta: 3`, with `current_count` moved to match (§13.3) — so the witness sits inside an array that is non-empty and the same length in both documents, where a presence reader and an emptiness reader are blind to the mutation. |
 | `witness-set-from-schema` | §13.2.1 step 2 reads the witness set from the **schema**, never from the document. The fixture carries a bare `vendor_private_counter` at the diff root, which is open: legal-but-undescribed, exit 0, and **not** a witness. Read the set from the document instead and it becomes one — a producer could then manufacture a witness by inventing a member at an open root. Measured 2026-09-01: that mutation passed the other twenty-five fixtures **25/25**, so until this entry existed the sentence had no arm at all. |
 | `pointer-token-literal-in-object` | The extension must not swallow the standard it extends. `-` means *every element* where an **array** sits, because RFC 6901 gives that token no resolvable meaning there; where an **object** sits it is a literal member name and stays one. This fixture is an envelope carrying a member spelled `-` beside a second member, so a reading that wildcards the token everywhere judges two documents where one was addressed. |
+| `withheld-signal-is-a-witness` | §13.2.2 lets a `"changed"` diff name, in `withheld_signals`, a finding it computed and did not serialise. The fixture is `changed_without_witness` with **one array's content** changed — `withheld_signals` from `[]` to `["field_histogram_deltas"]` — and it is CONFORMANT while its twin is not. Judge `withheld_signals` by presence rather than by its declared vacuity and both twins flip, which turns the escape into a hole any producer can climb through. |
+| `window-consistency-violation` | §2.2's relations between `window`'s members are unreachable from the schema, so every document here is **schema-valid** and wrong between its members: `duration_seconds` 300 over a 600-second window, `start` after `end`, an `end` with a legal non-UTC offset (also the withholding witness: with `end` unreadable, ordering and duration are not decided), and the §7 estimated-lines flag at `false`. |
+| `window-composed-envelope` | A composed document whose two raw children are **not contiguous** — 240 seconds of covered span inside a 600-second window. `duration_seconds` is `end - start`, 600; an arm that computed it from the children, or that treated the gap between shards as a defect, reds this conformant document. It is also the one positive witness that the estimated-lines flag admits `true`. |
+| `window-empty-extent` | §2.2: a window with **no line** (`lines_observed` 0) carries `start` equal to `end`. The fixture is one schema-valid document with `lines_observed` 0 whose `start` precedes its `end` by 600 seconds and whose `duration_seconds` is that 600, so ordering and duration both hold and this clause is the only thing that can catch it. Before the clause existed the validator read it CONFORMANT, exit 0. |
+| `window-empty-equal-instants` | The same clause's positive witness: two empty windows with `start` equal to `end`, the second spelling `end` as `…T10:00:00.000+00:00` against a `start` of `…T10:00:00Z` — one instant, two legal RFC 3339 spellings. A clause that compared the strings rather than the instants reds this conformant document; measured, that mutation reds exactly this fixture. |
 
 Measured on the committed tool, 2026-08-19: **seven** independent mutations each
 red the self-test — section-as-one-document · offending-member computation blinded ·
@@ -270,13 +275,16 @@ Declared, because an instrument's silence is read as coverage.
 
 - **§8 clause 2 is reached in ONE place, and clause 3 nowhere.** Clause 2 is *every
   required field populated according to its definition*, and for exactly one required
-  block that definition says something a schema cannot: `window` (§2.2) states three
+  block that definition says something a schema cannot: `window` (§2.2) states four
   relations BETWEEN its members — `start` at or before `end`, `duration_seconds`
-  equal to their difference to the nearest second, both instants in UTC — plus a §7
+  equal to their difference to the nearest second, both instants in UTC, and `start`
+  equal to `end` when `lines_observed` is 0 — plus a §7
   extension flag (`org.metalog.lines_observed_estimated`) that §2.2 admits only as
   `true` and that lives inside a container the schema deliberately leaves untyped.
-  Those four are decided here, on schema-valid documents only, and the withheld count
-  is printed. **Every other required field is still covered only as far as the schema
+  Those five are decided here, on schema-valid documents only, and the withheld count
+  is printed. What §2.2 relates to the window's LINES rather than to its members is
+  not: that `start` and `end` are the earliest and latest event time among the lines
+  the window contains needs those lines, which the document does not carry. **Every other required field is still covered only as far as the schema
   expresses it**, which is most of clause 2 — a `producer.name` that is the empty
   string, a `source.service` naming the wrong service, a `stats.frequency` that does
   not match its `count`: all schema-valid, none decided anywhere. The `window` arm

conformance/fixtures/invalid/window_empty_extent.metalog.jsonl

@@ -0,0 +1 @@
+{"metalog_version":"0.7.0","producer":{"name":"example-producer","version":"1.0.0"},"window":{"start":"2026-04-24T10:00:00Z","end":"2026-04-24T10:10:00Z","duration_seconds":600,"lines_observed":0},"source":{"service":"checkout"},"stats":{"unique_templates":0,"top_k_size":64,"tail_count":0,"tail_unique":0,"top_k":[]}}