spec(0.10.0): §2.4 — retention_profile names behavior.ngram_size; §12.1 omits behavior when two composed inputs differ in it

[coderoast-dev]

The problem

behavior.ngram_size is required (§4) and fixes what a top_ngrams key denotes: an order-2 key and an order-3 key are different objects. It is named by neither processing identifier in §2.4 — retention_profile lists top_k size, the reservoir parameters and the salience arithmetic; canonicalization_version lists masking, tokenization and classification. So two documents produced at different sequence orders can carry equal identifiers, and §2.4's comparability gate passes them. Three consequences:

• compose() breaks §12.2's commutativity MUST on a required field. A composer must declare one ngram_size; with A at 2 and B at 3, compose(A, B) and compose(B, A) declare different values.
• No minimum repairs it, unlike top_k_size / top_ngrams_size: those are ranking cuts over one population, while ngram_size selects a different population. Declaring the smaller order while carrying the larger order's sequence arrays is a false declaration; filtering to the smaller order drops one input's n-grams from a document whose window covers both, and no field reports the loss (dropped_ngram_observations counts observations refused at the producer). Counts across orders are also not comparable (every order-n occurrence implies an order-m one for m < n), and probability is conditioned to a different depth on each side.
• MetaLogDiff reports total turnover. §2.4's gate covers §13 in the same sentence, so the same equal-identifier pair is diffable: every n-gram of one side reads as vanished and every n-gram of the other as new. Under 0.10.0's §13.2 that maximally non-vacuous ngram_delta is a witness, so the document asserts "changed" for a change in configuration.

The proposed change

• §2.4 states what retention_profile names as a rule — every parameter that fixes which entries a bounded block retains, how they are ranked, and what an entry's key denotes — with the existing items and behavior.ngram_size as examples, the list illustrating the rule rather than closing it. The existing gate then refuses a cross-order compose() and MetaLogDiff at both sites, with no new mechanism.
• §12.1 gains the residual clause, because §2.4's gate binds only when both inputs carry the identifier: when both inputs carry behavior at different ngram_size, C.behavior MUST be omitted — never merged at a minimum, never carried from one side; when they agree, C.behavior.ngram_size is that value. The precondition is block-grained, like the cube clause; the rest of C composes as usual.
• The §1 table row and the schema's retention_profile description name the parameter (description text only; no keyword changed).
• CHANGELOG.md records it under 0.10.0 → Changed, adds it to the version's breaking summary, and notes that P3's item (f) is the first item of RFC rfc: 0.10.0 — close the extension boundary, make the diff's MUST decidable, and give the composed document its own caps #8's P3 taken (the caps rule stays proposed).

Classification, for the editor

Breaking by the letter: a producer keeping one retention_profile across two sequence orders, or a composer merging two orders' behavior blocks, becomes non-conformant. No document's validity changes: both identifiers stay opaque strings and behavior stays optional. Filed for merge under GOVERNANCE.md §2's 0.x rule; 0.10.0 has no tag and no Release.

Alternatives considered

• Take the minimum ngram_size, as for the other caps. Rejected: a rule that reads like its neighbours and means something else (see above).
• Fail the whole compose() on a mismatch. Rejected: the incomparability is confined to one optional block, and failing the document would destroy a well-defined stats merge. §2.4's hard failure stays document-grained, where the contract it guards is.
• Fix §12.1 only. Rejected: §13 would keep diffing two orders into a total-turnover witness. The missing axis is in the comparability contract; repairing it there repairs both operations.
• Carry the higher-order side's block alone. Rejected: it describes one input's lines under a window covering both.

Open question for the editor (not in this text)

§2.4's gate is conditional, so an identifier-less pair at different orders still reaches §13. The reference implementation emits no ngram_delta for such a pair (no key can match across orders). §13 says nothing about it today; whether it should state that, as §12.1 now does for compose(), is left to you.

Migration impact

The reference implementation (insight-metalog) adopts both rules in the same release: its derived retention_profile gains the sequence order as an axis, its compose() omits behavior across differing orders, and its diff emits no ngram_delta across them. Omitting the block also drops dropped_ngram_observations — a count lost, not falsified. conformance/metalog_validate.py --selftest: 32/32 fixtures pass on this branch.

🤖 Generated with Claude Code