spec(0.10.0): §4's probability is p(last | first n−1) among sequences of the entry's own length — "joint prob otherwise" withdrawn

CHANGELOG.md

@@ -36,7 +36,9 @@ The spec follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 **Breaking** under [`GOVERNANCE.md`](GOVERNANCE.md) §2 — a new **required** member
 on `MetaLogDiff`, a rewritten §13.2 clause, a `diff_version` rule (§13.1.1) that
 a producer stamping an older value now violates, and a definition of `window.start`
-and `window.end` (§2.2) that a producer writing its own bounds there now violates.
+and `window.end` (§2.2) that a producer writing its own bounds there now violates,
+and a definition of `behavior.top_ngrams[].probability` (§4) that a producer writing a
+joint probability at `ngram_size` above 2 now violates.
 It also adds one optional member
 (`withheld_signals`, §13.2.2), which is additive on its own. MINOR bump: MAJOR stays `0`, so
 §6's *"the MAJOR field of `metalog_version` must equal the MAJOR of the spec"* is
@@ -114,6 +116,23 @@ proposed; any of them that is taken lands as an editor change recorded here.
   into `start`/`end` moves them to an extension (§7). RFC: pull request
   [#14](https://github.com/CodeRoasted/metalog-spec/pull/14).
 
+- **§4 — `top_ngrams[].probability` is p(last | first n − 1) at every `ngram_size`, among
+  sequences of the entry's own length; the *"joint prob otherwise"* text is withdrawn.**
+  The example's comment read *"p(next | prev) for n=2; joint prob otherwise"*, the only
+  definition the field had. It was wrong against the one shipped producer, which has
+  always emitted the conditional at every order, and wrong against the field's one
+  consumer in this specification: §13's `ngram_delta.rate_changed` compares the value
+  across two windows as a transition rate, and a joint value restates frequency, which
+  `count` already carries, so it would move with any change in traffic mix. A new
+  paragraph under §4 now defines the value: the entry's `count` over the summed `count`
+  of the window's counted sequences of the same length sharing its first n − 1 ids,
+  computed before the `top_ngrams_size` cut. The "same length" clause also fixes a
+  defect found in the reference implementation, which pooled sequences of two lengths
+  under one denominator when its `top_ngrams` carried both; the fix moves only such
+  documents. **Breaking by the letter** — a producer that followed "joint" becomes
+  non-conformant — and editor-merged under [`GOVERNANCE.md`](GOVERNANCE.md) §2's 0.x
+  rule. No schema changed: the field was, and stays, a number in [0, 1].
+
 ### Added
 
 - **§13.2.1 — `x-metalog-vacuous`, a per-property vacuity declaration.** The witness

SPEC.md

@@ -801,7 +801,7 @@ Captures *how* templates follow each other, beyond raw frequency.
     {
       "sequence": ["h:8a3f...", "h:b104..."],  // array of template_ids
       "count": 8421,
-      "probability": 0.677              // p(next | prev) for n=2; joint prob otherwise
+      "probability": 0.677              // p(last | first n−1), among sequences of this length — see below
     }
   ],
   "top_ngrams_size": 64,                // integer, required
@@ -821,6 +821,19 @@ Captures *how* templates follow each other, beyond raw frequency.
 }
 ```
 
+**`probability` — a conditional, never a joint.** An entry's `probability` is
+p(last | prefix): its `count` divided by the summed `count` of every sequence **of the
+same length** the producer counted in the window whose first n − 1 template ids equal
+the entry's own, n being the length of the entry's `sequence`. At `ngram_size` 2 it is
+p(next | prev); at 3 it is p(third | first two). It is computed over every counted
+sequence, **before** the `top_ngrams_size` cut, so the retained entries sharing a
+prefix need not sum to 1. A sequence of another length never enters the sum: in a
+`top_ngrams` holding sequences of more than one length, each length is conditioned
+among its own. It is a conditional and not a joint probability because frequency is
+already carried by `count`, and because §13's `ngram_delta.rate_changed` compares this
+value across two windows as a transition RATE: a joint value would move with any change
+in traffic mix and would mean a different quantity at each `ngram_size`.
+
 **`dropped_ngram_observations` — the accounting bound, distinct from top-k truncation.**
 `top_ngrams` is a RANKING cut: every entry it drops was seen, counted and ranked, and the
 retained set is the top `top_ngrams_size` of them. A producer MAY additionally bound the