From 0e65dec87c2b6b3ea316f72309cb55b35018ece6 Mon Sep 17 00:00:00 2001 From: Emmanuel Prunet Date: Tue, 29 Sep 2026 22:10:41 +0200 Subject: [PATCH 1/2] spec(0.10.0): window.start and window.end are the event-time envelope of the window's lines, and how a producer decides a window's extent is its own MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The two instants had no definition. A producer could write the bounds its own rule used, or the times of the lines it read, and a consumer could not tell which; §12.1 takes their min and max, so two documents could compose into an interval that meant neither. §2.2 now defines them as the earliest and the latest event time among the lines the window contains, requires start equal to end for a window with no line, and states that which lines a window contains, and how and when a producer decides it, is implementation-defined: no fixed interval, no decision on a single line, no online decision. The glossary gains Event time and redefines Window. The §1 table, §5 (previous_window_end is the previous window's window.end), §8 (four window relations), §12.1, §16.6 (the regime note becomes a MUST NOT, and loses a tag nothing defined), §16.9, §16.10 and RATIONALE §R5 drop the wording that assumed a fixed interval closed as lines arrive. Breaking under GOVERNANCE.md §2; this pull request is its RFC. Co-Authored-By: Claude Opus 5.5 (1M context) --- CHANGELOG.md | 28 +++++++++++++++++--- RATIONALE.md | 4 +-- SPEC.md | 73 ++++++++++++++++++++++++++++++++++++---------------- 3 files changed, 78 insertions(+), 27 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 167b059..a1cb10a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -34,8 +34,10 @@ The spec follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html). ## [0.10.0] — UNRELEASED **Breaking** under [`GOVERNANCE.md`](GOVERNANCE.md) §2 — a new **required** member -on `MetaLogDiff`, a rewritten §13.2 clause, and a `diff_version` rule (§13.1.1) that -a producer stamping an older value now violates. It also adds one optional member +on `MetaLogDiff`, a rewritten §13.2 clause, a `diff_version` rule (§13.1.1) that +a producer stamping an older value now violates, and a definition of `window.start` +and `window.end` (§2.2) that a producer writing its own bounds there now violates. +It also adds one optional member (`withheld_signals`, §13.2.2), which is additive on its own. MINOR bump: MAJOR stays `0`, so §6's *"the MAJOR field of `metalog_version` must equal the MAJOR of the spec"* is satisfied unchanged and **both schema files keep their `v0` filenames** — a reader @@ -53,7 +55,10 @@ no implementer to break. Folding them here rather than minting 0.11.0 also keeps one number across `SPEC.md`, `metalog_version` and `diff_version`, which is the whole point of the first of them. **The size headline withdrawn under *Removed* is not from that RFC either**: it is an editorial change, raised by the editor after a -measurement, and it changes no document's validity. +measurement, and it changes no document's validity. **Nor is the §2.2 window +definition under *Changed***: it is a breaking change of its own, proposed and argued +in pull request [#14](https://github.com/CodeRoasted/metalog-spec/pull/14) under its own +14-day comment window. ### Changed @@ -85,6 +90,23 @@ measurement, and it changes no document's validity. stated rather than hidden: **§13.2 is no longer readable standalone**, and a reader must consult the schema to know the set. +- **`window.start` and `window.end` are defined, and the specification stops assuming + how a producer decides a window's extent (§2.2).** The two instants had no + definition: a producer could write the bounds its own rule used or the times of the + lines it read, and a consumer could not tell which — so §12.1, which takes their + `min` and `max`, could compose two documents into an interval that meant neither. + They are now the earliest and the latest **event time** (a new glossary term) among + the lines the window contains, and a window with no line carries `start` equal to + `end`. Which lines a window contains, and how and when a producer decides it, is + implementation-defined; a producer **MAY** fix the extent only after reading beyond + it. The glossary, the §1 table, §5, §12.1, §16.6, §16.9, §16.10 and RATIONALE §R5 + drop the wording that assumed a fixed interval closed as lines arrive: §5's + `previous_window_end` is the previous window's `window.end`, and §16.6's regime + note becomes a **MUST NOT**. §8 counts the empty-window equality among the window + relations the conformance validator decides. A producer that writes its own bounds + into `start`/`end` moves them to an extension (§7). RFC: pull request + [#14](https://github.com/CodeRoasted/metalog-spec/pull/14). + ### Added - **§13.2.1 — `x-metalog-vacuous`, a per-property vacuity declaration.** The witness diff --git a/RATIONALE.md b/RATIONALE.md index 94360e9..32509e6 100644 --- a/RATIONALE.md +++ b/RATIONALE.md @@ -180,8 +180,8 @@ pressure handles the rest. Doesn't compose. The producer chooses what "previous window" means (typically the -immediately preceding window of the same duration) and reports its -boundary in `previous_window_end`. +immediately preceding window) and names it in `previous_window_end` +by that window's `window.end`. --- diff --git a/SPEC.md b/SPEC.md index 2488288..de9399d 100644 --- a/SPEC.md +++ b/SPEC.md @@ -45,8 +45,13 @@ keywords: **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT**, **MAY**. `User <*> logged in from <*>`. - **TemplateID** — A stable, content-derived identifier for a template. See §3.2. -- **Window** — A contiguous time interval over which a single MetaLog - is computed. +- **Window** — The log lines a single MetaLog is computed over. How a + producer decides which lines a window contains is + implementation-defined (§2.2). +- **Event time** — The time a producer attributes to a log line: the + timestamp the line carries, when it carries one the producer can + read. How a producer attributes a time to a line that carries none + is implementation-defined. - **Producer** — A program that consumes log lines and emits MetaLog documents. - **Consumer** — A program that reads MetaLog documents (dashboard, @@ -70,7 +75,7 @@ top-level fields: |---|---|---|---| | `metalog_version` | string | yes | Spec version this document conforms to. SemVer string (e.g. `"0.10.0"`). The `MetaLogDiff` (§13) carries the same axis as `diff_version` — see §13.1.1. | | `producer` | object | yes | Identifies the producing implementation. See §2.1. | -| `window` | object | yes | The time interval covered. See §2.2. | +| `window` | object | yes | The event-time envelope of the lines the window contains. See §2.2. | | `source` | object | yes | What was observed (service, host, fleet). See §2.3. | | `canonicalization_version` | string | no | Opaque identifier for the canonicalization rules in effect. Gates `compose()`/diff comparability. See §2.4. | | `retention_profile` | string | no | Opaque identifier for the retention parameters (top_k, reservoir, salience weights, diversity caps). Gates `compose()`/diff comparability. See §2.4. | @@ -166,8 +171,30 @@ that as a coupling. } ``` -`start` **MUST** be strictly less than or equal to `end` -(equality is permitted for empty / heartbeat documents). +**How a producer decides a window's extent is outside this +specification.** Which lines a window contains, and how and when the +producer decides it, are implementation-defined, and this +specification assumes no producer behaviour for that decision. In +particular it assumes neither a fixed interval, nor a decision taken +on any single line, nor an online or causal decision: a producer +**MAY** fix a window's extent only after reading lines beyond it, and +**MAY** assign lines to a window retrospectively. A document that +carries a raw re-derivation coordinate still describes exactly the +lines whose event time falls within that coordinate's `bounds` +(§15.3). + +`start` and `end` are the earliest and the latest event time among +the lines the window contains: the window's event-time envelope. They +say nothing about when or how the window's extent was decided, nor +about when the document was emitted. A producer that has bounds of +its own — the interval its rule for deciding the extent used — +**MAY** publish them in an extension (§7). + +`start` **MUST** be less than or equal to `end`; the two are equal +when every line the window contains carries the same event time, a +window of one line included. For a window containing no line +(`lines_observed` 0), `start` **MUST** equal `end`, and both name the +time the producer attributes to the window. `duration_seconds` **MUST** equal `end - start` rounded to the nearest second. If a producer cannot count `lines_observed` exactly, it **MUST** emit its best estimate and **SHOULD** emit an @@ -867,11 +894,12 @@ entirely rather than emit empty fields. Quantifies *how much the system's behaviour changed* since the previous window. Stability is a special case of §13 Diff with the -`previous` document being the previous closed window. +`previous` document being the window before it in the producer's +order. ```jsonc { - "previous_window_end": "2026-04-24T10:00:00Z", // RFC 3339, required + "previous_window_end": "2026-04-24T10:00:00Z", // RFC 3339, required, the previous window's `window.end` "kl_divergence": 0.043, // number ≥ 0, KL(current || previous) over template freqs "js_divergence": 0.021, // number in [0, 1], symmetric Jensen-Shannon "new_templates": 3, // integer, templates seen now but not in previous window @@ -1032,10 +1060,11 @@ the file. Clause 2 is mechanically decidable in exactly one place. `window` (§2.2) is the only required block whose definition states relations **between** its members — `start` at or before `end`, `duration_seconds` equal to their difference rounded to the nearest -second, and both instants in UTC. None of the three is expressible -in JSON Schema at any draft (`format: date-time` constrains the -grammar, never the offset, and no keyword relates two siblings), -and all three follow from the document alone, so +second, both instants in UTC, and `start` equal to `end` when +`lines_observed` is 0. None of the four is expressible in JSON +Schema at any draft (`format: date-time` constrains the grammar, +never the offset, and no keyword relates two siblings), and all four +follow from the document alone, so [`conformance/metalog_validate.py`](conformance/metalog_validate.py) decides them — together with the §7 `org.metalog.lines_observed_estimated` flag, which §2.2 admits only @@ -1269,7 +1298,7 @@ to a 1-hour MetaLog). - `C.window.start = min(A.window.start, B.window.start)` - `C.window.end = max(A.window.end, B.window.end)` -- `C.window.duration_seconds = C.window.end - C.window.start` (real time, **not** sum of inputs) +- `C.window.duration_seconds = C.window.end - C.window.start` (the event-time envelope of both inputs' lines, **not** the sum of their durations) - `C.window.lines_observed = A.window.lines_observed + B.window.lines_observed` - `C.source` is `A.source` if equal to `B.source`, otherwise the most-specific common prefix (e.g. same `fleet`, drop differing @@ -2235,11 +2264,10 @@ This cross is a **hard firewall (normative)**: entry with its cube coordinate. - `cube_coord` is a **pure function** of the entry's `(level, where-path)` — no new non-determinism, no float. -- **Regime precondition (D9):** valid **only** while the reservoir and the cube close - on the **same window boundary** (the fixed-window regime — true today). Under - adaptive window closure the two may close on different boundaries, so the cross - would point at a cell of the wrong window; it **MUST** be re-designed before reuse - there (reuse unchanged is *out-of-regime*, not a bug). +- **Regime precondition:** valid **only** while the reservoir and the cube are + computed over the same lines, one window's content. A producer whose reservoir and + cube can cover different lines **MUST NOT** emit `cube_coord`: the cross would + point at a cell computed over other lines. ### 16.7 Two scales — intra-window and compose @@ -2272,9 +2300,10 @@ the source of truth for any categorical marginal and does **not** reshape `stats ### 16.9 Determinism -The cube is computed **in batch over the closed window** — a finite, frozen, ordered -set — so it is a **pure function** of that set and **bit-identical across stdlibs and -operating systems** (the standing cross-stdlib / cross-OS diagonal). Specifically: +The cube is computed **in batch over the window's final set of lines** — a finite, +frozen, ordered set — so it is a **pure function** of that set and **bit-identical +across stdlibs and operating systems** (the standing cross-stdlib / cross-OS +diagonal). Specifically: - `count` is integer; the closure and the border are set operations; cells and border cells **MUST** serialise in canonical (coord-sorted) order. @@ -2284,7 +2313,7 @@ operating systems** (the standing cross-stdlib / cross-OS diagonal). Specificall carries **one** `canonicalization_version` bump and its golden cascade in the same pass. - The §16.10 collapse is **deterministic content**: the trigger reads only the - closed window (never wall-clock, never an environment budget); the policy is pure + window's lines (never wall-clock, never an environment budget); the policy is pure integer (`Δcells / cost` by cross-multiplication — no float); and the fixed candidate order (LEVEL before WHERE) **is** the declared total-order tie-break. The policy is **version-stamped** — changing its budget, costs, steps or @@ -2304,7 +2333,7 @@ budget-driven dimensional collapse** — three separated objects: declaring it is what makes it knowable to the **consumer**. It dominates every other term of the size formula (§11.4), so an undeclared budget leaves the consumer unable to price the block at all. -- **TRIGGER** — a pure function of the closed window's content: closed cells over +- **TRIGGER** — a pure function of the window's content: closed cells over budget. **Closure-first, collapse-last**: when closure alone fits, nothing degrades — collapse is the rare guard, not the normal path. - **POLICY** — while over budget, apply the best **admissible** monotone coarsening From e113f115ce89c2b5e23a0caa320713ae41fab2d0 Mon Sep 17 00:00:00 2001 From: Emmanuel Prunet Date: Tue, 29 Sep 2026 22:31:37 +0200 Subject: [PATCH 2/2] =?UTF-8?q?conformance:=20the=20validator=20decides=20?= =?UTF-8?q?=C2=A72.2's=20empty-window=20relation=20=E2=80=94=20a=20window?= =?UTF-8?q?=20with=20no=20line=20has=20`start`=20equal=20to=20`end`,=20jud?= =?UTF-8?q?ged=20on=20the=20instants?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §8 now counts that equality among the window relations this validator decides, and until this commit nothing decided it. A schema-valid document with `lines_observed` 0 whose `start` precedes its `end` by 600 seconds, and whose `duration_seconds` is 600, read CONFORMANT, exit 0: ordering and duration both hold, so no existing clause could see it. - metalog_validate.py: a fifth window clause, `empty-extent`, decided apart from ordering and on the parsed instants, never on the strings. RFC 3339 spells one instant several ways, and §2.2 constrains the time, not its spelling. The printed scope names the clause, and also names what the window arm cannot decide now that §2.2 defines `start` and `end` as the event-time envelope of the window's lines: a relation to lines the document does not carry. - invalid/window_empty_extent.metalog.jsonl (control `window-empty-extent`): the must-fail document above. Red first, against the validator before this commit: the self-test with this manifest entry reads `SELFTEST FAILED: 1/31 fixtures — ['invalid/window_empty_extent.metalog.jsonl']`, and the fixture validated alone reads `VERDICT: CONFORMANT`, exit 0. - valid/window_empty_equal_instants.metalog.jsonl (control `window-empty-equal-instants`): the clause's positive witness, two empty windows with `start` equal to `end`, the second spelling `end` as `2026-04-24T10:00:00.000+00:00` against a `start` of `2026-04-24T10:00:00Z`. A clause that compares the strings reds it, and only it: `SELFTEST FAILED: 1/31 fixtures — ['valid/window_empty_equal_instants.metalog.jsonl']`. - README: the counts (thirty-one fixtures, nineteen tagged, seventeen distinct controls), a table row for every control the self-test arms and the table did not list (five, three of them already armed before this commit), and the clause-2 limit restated with the fifth relation and the envelope it cannot reach. After: `SELFTEST PASSED: 31/31 fixtures`, seventeen controls armed; the shipped example reads CONFORMANT with `--expect-documents 1`. Co-Authored-By: Claude Opus 5.5 (1M context) --- conformance/README.md | 20 +++++++--- .../invalid/window_empty_extent.metalog.jsonl | 1 + conformance/fixtures/manifest.json | 36 +++++++++++++++++- .../window_empty_equal_instants.metalog.jsonl | 2 + conformance/metalog_validate.py | 37 +++++++++++++++---- 5 files changed, 81 insertions(+), 15 deletions(-) create mode 100644 conformance/fixtures/invalid/window_empty_extent.metalog.jsonl create mode 100644 conformance/fixtures/valid/window_empty_equal_instants.metalog.jsonl diff --git a/conformance/README.md b/conformance/README.md index a5828f9..5adf467 100644 --- a/conformance/README.md +++ b/conformance/README.md @@ -144,13 +144,13 @@ never looked. ## Why the self-test exists -A validator that cannot fail is decoration. `--selftest` runs twenty-six fixtures whose +A validator that cannot fail is decoration. `--selftest` runs thirty-one fixtures whose expected results are **hand-authored in `fixtures/manifest.json` from the spec and the schemas** — never captured from a run, because an expectation copied out of the tool under test makes the tool its own oracle, and the pair then agree forever while both are wrong. -Fourteen of those fixtures carry a `control` tag, naming **twelve** distinct blindnesses +Nineteen of those fixtures carry a `control` tag, naming **seventeen** distinct blindnesses (`instrument-failure` is carried by three), and the self-test **refuses to run** if any tag is missing from the manifest — deleting a fixture cannot quietly widen what a green covers. Each control forecloses one specific way this validator could @@ -170,6 +170,11 @@ have gone green while blind: | `witness-rule-unchanged-arm` | `"unchanged"` forbids one, and a rule that binds only the other arm is satisfied forever by a producer that always writes `"unchanged"`. The fixture is the same control with **one row** changed — `template_deltas[0]` from `delta: 0` to `delta: 3`, with `current_count` moved to match (§13.3) — so the witness sits inside an array that is non-empty and the same length in both documents, where a presence reader and an emptiness reader are blind to the mutation. | | `witness-set-from-schema` | §13.2.1 step 2 reads the witness set from the **schema**, never from the document. The fixture carries a bare `vendor_private_counter` at the diff root, which is open: legal-but-undescribed, exit 0, and **not** a witness. Read the set from the document instead and it becomes one — a producer could then manufacture a witness by inventing a member at an open root. Measured 2026-09-01: that mutation passed the other twenty-five fixtures **25/25**, so until this entry existed the sentence had no arm at all. | | `pointer-token-literal-in-object` | The extension must not swallow the standard it extends. `-` means *every element* where an **array** sits, because RFC 6901 gives that token no resolvable meaning there; where an **object** sits it is a literal member name and stays one. This fixture is an envelope carrying a member spelled `-` beside a second member, so a reading that wildcards the token everywhere judges two documents where one was addressed. | +| `withheld-signal-is-a-witness` | §13.2.2 lets a `"changed"` diff name, in `withheld_signals`, a finding it computed and did not serialise. The fixture is `changed_without_witness` with **one array's content** changed — `withheld_signals` from `[]` to `["field_histogram_deltas"]` — and it is CONFORMANT while its twin is not. Judge `withheld_signals` by presence rather than by its declared vacuity and both twins flip, which turns the escape into a hole any producer can climb through. | +| `window-consistency-violation` | §2.2's relations between `window`'s members are unreachable from the schema, so every document here is **schema-valid** and wrong between its members: `duration_seconds` 300 over a 600-second window, `start` after `end`, an `end` with a legal non-UTC offset (also the withholding witness: with `end` unreadable, ordering and duration are not decided), and the §7 estimated-lines flag at `false`. | +| `window-composed-envelope` | A composed document whose two raw children are **not contiguous** — 240 seconds of covered span inside a 600-second window. `duration_seconds` is `end - start`, 600; an arm that computed it from the children, or that treated the gap between shards as a defect, reds this conformant document. It is also the one positive witness that the estimated-lines flag admits `true`. | +| `window-empty-extent` | §2.2: a window with **no line** (`lines_observed` 0) carries `start` equal to `end`. The fixture is one schema-valid document with `lines_observed` 0 whose `start` precedes its `end` by 600 seconds and whose `duration_seconds` is that 600, so ordering and duration both hold and this clause is the only thing that can catch it. Before the clause existed the validator read it CONFORMANT, exit 0. | +| `window-empty-equal-instants` | The same clause's positive witness: two empty windows with `start` equal to `end`, the second spelling `end` as `…T10:00:00.000+00:00` against a `start` of `…T10:00:00Z` — one instant, two legal RFC 3339 spellings. A clause that compared the strings rather than the instants reds this conformant document; measured, that mutation reds exactly this fixture. | Measured on the committed tool, 2026-08-19: **seven** independent mutations each red the self-test — section-as-one-document · offending-member computation blinded · @@ -270,13 +275,16 @@ Declared, because an instrument's silence is read as coverage. - **§8 clause 2 is reached in ONE place, and clause 3 nowhere.** Clause 2 is *every required field populated according to its definition*, and for exactly one required - block that definition says something a schema cannot: `window` (§2.2) states three + block that definition says something a schema cannot: `window` (§2.2) states four relations BETWEEN its members — `start` at or before `end`, `duration_seconds` - equal to their difference to the nearest second, both instants in UTC — plus a §7 + equal to their difference to the nearest second, both instants in UTC, and `start` + equal to `end` when `lines_observed` is 0 — plus a §7 extension flag (`org.metalog.lines_observed_estimated`) that §2.2 admits only as `true` and that lives inside a container the schema deliberately leaves untyped. - Those four are decided here, on schema-valid documents only, and the withheld count - is printed. **Every other required field is still covered only as far as the schema + Those five are decided here, on schema-valid documents only, and the withheld count + is printed. What §2.2 relates to the window's LINES rather than to its members is + not: that `start` and `end` are the earliest and latest event time among the lines + the window contains needs those lines, which the document does not carry. **Every other required field is still covered only as far as the schema expresses it**, which is most of clause 2 — a `producer.name` that is the empty string, a `source.service` naming the wrong service, a `stats.frequency` that does not match its `count`: all schema-valid, none decided anywhere. The `window` arm diff --git a/conformance/fixtures/invalid/window_empty_extent.metalog.jsonl b/conformance/fixtures/invalid/window_empty_extent.metalog.jsonl new file mode 100644 index 0000000..ca42d47 --- /dev/null +++ b/conformance/fixtures/invalid/window_empty_extent.metalog.jsonl @@ -0,0 +1 @@ +{"metalog_version":"0.7.0","producer":{"name":"example-producer","version":"1.0.0"},"window":{"start":"2026-04-24T10:00:00Z","end":"2026-04-24T10:10:00Z","duration_seconds":600,"lines_observed":0},"source":{"service":"checkout"},"stats":{"unique_templates":0,"top_k_size":64,"tail_count":0,"tail_unique":0,"top_k":[]}} diff --git a/conformance/fixtures/manifest.json b/conformance/fixtures/manifest.json index 5299f4a..3c67070 100644 --- a/conformance/fixtures/manifest.json +++ b/conformance/fixtures/manifest.json @@ -83,6 +83,40 @@ "window_unjudged": 0 } }, + { + "path": "invalid/window_empty_extent.metalog.jsonl", + "kind": "metalog", + "control": "window-empty-extent", + "why": "SPEC §2.2: a window containing no line (`lines_observed` 0) carries `start` equal to `end`, and §8 counts that equality among the window relations this validator decides. One SCHEMA-VALID document with `lines_observed` 0 whose `start` precedes its `end` by 600 seconds, and whose `duration_seconds` is that 600, so the ordering and duration clauses both hold and the empty-extent clause is the only thing that can catch it. Before this clause existed the validator read this document CONFORMANT, exit 0.", + "expect": { + "exit": 1, + "documents": 1, + "findings": [], + "undescribed": [], + "window_violations": [ + [ + "empty-extent", + "lines_observed is 0 and start 2026-04-24T10:00:00Z is not end 2026-04-24T10:10:00Z", + 1 + ] + ], + "window_unjudged": 0 + } + }, + { + "path": "valid/window_empty_equal_instants.metalog.jsonl", + "kind": "metalog", + "control": "window-empty-equal-instants", + "why": "The empty-extent clause's positive witness: two SCHEMA-VALID documents with `lines_observed` 0 and `start` equal to `end`, which §2.2 admits and which MUST read CONFORMANT. The second spells `end` as `2026-04-24T10:00:00.000+00:00`, the same instant as its `start` `2026-04-24T10:00:00Z` in another legal RFC 3339 spelling, so a clause that compared the two strings rather than the two instants reds here. A clause tested only by its violation passes forever as `always red on an empty window`.", + "expect": { + "exit": 0, + "documents": 2, + "findings": [], + "undescribed": [], + "window_violations": [], + "window_unjudged": 0 + } + }, { "path": "undescribed/open_containers.metalog.jsonl", "kind": "metalog", @@ -567,6 +601,6 @@ } } ], - "window_violations_encoding": "[clause, detail, documents affected]. SPEC §8 clause 2, over §2.2's `window` and nothing else. `clause` is one of `utc` / `ordering` / `duration` / `estimated-flag`; `detail` is the instrument's own sentence for that violation and is asserted VERBATIM, because the four clauses are decided by four different computations and a fixture pinning only the clause name would pass against an arm that fired the right label off the wrong arithmetic. Types are NOT among these clauses: `duration_seconds` and `lines_observed` are typed `integer, minimum 0` by the schema, so their non-negativity is clause 1's business and re-checking it here would be a second copy that can disagree with the first.", + "window_violations_encoding": "[clause, detail, documents affected]. SPEC §8 clause 2, over §2.2's `window` and nothing else. `clause` is one of `utc` / `ordering` / `duration` / `empty-extent` / `estimated-flag`; `detail` is the instrument's own sentence for that violation and is asserted VERBATIM, because the five clauses are decided by five different computations and a fixture pinning only the clause name would pass against an arm that fired the right label off the wrong arithmetic. Types are NOT among these clauses: `duration_seconds` and `lines_observed` are typed `integer, minimum 0` by the schema, so their non-negativity is clause 1's business and re-checking it here would be a second copy that can disagree with the first.", "window_unjudged_encoding": "how many documents §2.2's relations were WITHHELD from, because they are not schema-valid. Defaults to 0 and is compared on every non-refusal fixture, the same discipline `witness_unjudged` carries and for the same reason: on an unvalidated document `start` may be an integer and `duration_seconds` a string, so judging one would report an arithmetic defect at a producer whose real defect is a type. A withheld verdict is not a pass, and leaving it unasserted would let an arm that stopped judging anything pass this suite green. Nonzero exactly on the metalog fixtures that carry a schema violation." } diff --git a/conformance/fixtures/valid/window_empty_equal_instants.metalog.jsonl b/conformance/fixtures/valid/window_empty_equal_instants.metalog.jsonl new file mode 100644 index 0000000..166864d --- /dev/null +++ b/conformance/fixtures/valid/window_empty_equal_instants.metalog.jsonl @@ -0,0 +1,2 @@ +{"metalog_version":"0.7.0","producer":{"name":"example-producer","version":"1.0.0"},"window":{"start":"2026-04-24T10:00:00Z","end":"2026-04-24T10:00:00Z","duration_seconds":0,"lines_observed":0},"source":{"service":"checkout"},"stats":{"unique_templates":0,"top_k_size":64,"tail_count":0,"tail_unique":0,"top_k":[]}} +{"metalog_version":"0.7.0","producer":{"name":"example-producer","version":"1.0.0"},"window":{"start":"2026-04-24T10:00:00Z","end":"2026-04-24T10:00:00.000+00:00","duration_seconds":0,"lines_observed":0},"source":{"service":"checkout"},"stats":{"unique_templates":0,"top_k_size":64,"tail_count":0,"tail_unique":0,"top_k":[]}} diff --git a/conformance/metalog_validate.py b/conformance/metalog_validate.py index e6fc996..e22c42a 100644 --- a/conformance/metalog_validate.py +++ b/conformance/metalog_validate.py @@ -710,6 +710,11 @@ def walk(node, path: str, index: int, depth: int): # duration — `duration_seconds` MUST equal `end - start` rounded to the # nearest second. Also a relation, and the one a producer gets # wrong silently: every member is individually well-typed. +# empty-extent — a window containing no line (`lines_observed` 0) MUST carry +# `start` equal to `end`. A relation among three members, and +# decided on the two INSTANTS, never on the two strings: RFC 3339 +# spells one instant several ways (`Z`, `+00:00`, a fractional +# `.000`), and §2.2 constrains the time, not its spelling. # estimated-flag — `extensions.org.metalog.lines_observed_estimated`, when # present, is `true`. It lives inside §7's OPEN extension # container, whose whole contract is that the schema does not @@ -851,6 +856,12 @@ def note(clause: str, detail: str, label: str) -> None: note("duration", f"duration_seconds declares {declared}, " f"end - start is {delta:g}", label) + # Decided apart from ordering: a window with no line and `start` after + # `end` breaks both relations, and each is reported by its own clause. + if window.get("lines_observed") == 0 and start != end: + note("empty-extent", + f"lines_observed is 0 and start {window['start']} is not end " + f"{window['end']}", label) flag = _estimated_flag(doc) if flag is not _ABSENT and flag is not True: @@ -1190,6 +1201,7 @@ def render(report: dict, stream) -> None: f"{acc['documents']} {'carries a' if judged == 1 else 'carry'} " f"`{WINDOW_MARKER}` whose start and end are RFC 3339 UTC instants in " f"order, whose `duration_seconds` is `end - start` to the nearest second, " + f"whose start equals its end when it contains no line, " f"and whose §7 estimated-lines flag, where present, is `true`.") if report["window_governed"] and report["window_unjudged"]: w(f" NOT judged: {plural(report['window_unjudged'], 'document')} — §2.2's " @@ -1229,18 +1241,21 @@ def render(report: dict, stream) -> None: w(" Clause 2's own limit, and it is narrow: `window` is the one required block") w(" whose definition states relations BETWEEN its members, so it is the one") w(" part of clause 2 a reader can decide from the document alone. Every other") - w(" required field is checked only as far as the schema expresses it. The four") + w(" required field is checked only as far as the schema expresses it. The five") w(" relations decided here are the UTC-ness of `start` and `end` (`format:") w(" date-time` accepts any offset, and by default asserts nothing at all),") - w(" their ordering, `duration_seconds` against `end - start`, and the §7") - w(" estimated-lines flag. Their TYPES are clause 1's business and are not") - w(" re-checked here. §2.2 states UTC as a field definition rather than with the") - w(" word MUST; this tool reads a definition as binding, so a `+02:00` offset is") - w(" reported — open §2.2 before treating that as a producer bug rather than a") - w(" spec-prose question.") + w(" their ordering, `duration_seconds` against `end - start`, their equality") + w(" when `lines_observed` is 0, and the §7 estimated-lines flag. Their TYPES") + w(" are clause 1's business and are not re-checked here. §2.2 states UTC as a") + w(" field definition rather than with the word MUST; this tool reads a") + w(" definition as binding, so a `+02:00` offset is reported — open §2.2 before") + w(" treating that as a producer bug rather than a spec-prose question.") w(" NOT checked: the rest of clause 2, and clause 3 (template_id computed") w(" per §3.2 — no pinned cross-implementation vector exists yet). A green above") - w(" says nothing about those two.") + w(" says nothing about those two. The rest of clause 2 includes part of") + w(" `window` itself: that `start` and `end` ARE the earliest and latest event") + w(" time among the window's lines is a relation to lines the document does") + w(" not carry, and no reader of the document alone can decide it.") w(" Clause 4's own limit: a cap that is not DECLARED cannot be checked. A") w(" producer that omits `behavior.branching_size` declares no cap (§4.2), and") w(" this tool reads that as a posture, never as a pass.") @@ -1498,6 +1513,12 @@ def census_control() -> None: # composed document's CHILDREN instead of from # its own start/end, which reds a conformant # composition across every gap between shards + "window-empty-extent", # forecloses can't-FAIL on §2.2's empty-window + # relation: `start` equal to `end` when the + # window contains no line + "window-empty-equal-instants", # forecloses the same relation decided on the + # two STRINGS, which reds a conformant empty + # window whose instants are spelled apart "withheld-signal-is-a-witness", # forecloses a §13.2.2 escape that cannot be # taken: the whole point of `withheld_signals` # is that a NON-EMPTY one carries the outcome,