Skip to content

fix: ground retention references in structured turn authors - #243

Merged
tenphi merged 1 commit into
mainfrom
fix/context-witness-grounding
Oct 18, 2026
Merged

tenphi merged 1 commit into
mainfrom
fix/context-witness-grounding

Conversation

@tenphi

@tenphi tenphi commented Oct 18, 2026

Copy link
Copy Markdown
Owner

Summary

The automatic-retention context witness was rejecting two kinds of legitimate references as context_witness_invalid. Each rejection held the candidate as discourse_uncertain. While measuring #241, these were the largest single loss channel.

  1. Speakerless authors. Assistant turns, and user turns without a speaker, have no author label. The verifier's first-person reference (source_name: "assistant", explicit, own frame) could therefore never be grounded.
  2. Earlier-turn antecedents. The pattern is "I, Ada Marlow, declined the optional Zephyr QX-100 cable-protection extension." in one turn, then "Had I accepted that extension …" in the next. The rule from fix: retain source context through immutable antecedent witnesses #226 that name origins come only from the same item rejected every such binding.

What changes:

  • retentionItemAuthor gives an item its speaker, or its assistant/user role when it has no speaker. The existing own-frame first-person rule then applies unchanged.
  • retentionContextWitness takes the source's item order. An unambiguous_antecedent may now anchor on an exact name in an earlier item with the identical author label.
  • The verifier contract describes both. RETAIN_VERIFIER_VERSION advances to v54, as fix: retain source context through immutable antecedent witnesses #226 did when it changed this contract.

Still rejected, with a test for each:

  • another author's turn;
  • a later turn;
  • unknown source order;
  • items without an author label;
  • explicit across frames;
  • a role label used by another frame;
  • a preserved role or name that is missing from the record.

Every reference keeps its semantic-role, preservation and completeness checks. The inner-reporter attribution audit keeps its same-item rule.

Measurement: an exact replay with identical model responses

I replayed all 224 live gpt-6-luna retention runs recorded for #241 through runRetain. A model stub returned each recorded response, so every local check ran fresh under the old and the new grounding. 221 of 224 runs replayed completely; 3 revision calls had no recorded counterpart and were treated as unavailable.

Old grounding New grounding
Written memories 216 251 (+35)
context_witness_invalid holds 87 34
Previously written memories lost — 0
Shipped config (single extraction + #241 revision): required recall 56.1% 62.8%
Shipped config: written / held 105 / 93 120 / 78
Shipped config: judged faithful precision 90.5% 88.3%

I hand-audited the 35 newly admitted records against their sources, and so did a blinded gpt-6-luna judge.

  • All 35 have supported, correctly qualified text, and none shows an identity misbinding.
  • The judge flagged 7 for labels that come from extraction:
    • 5 are assistant statements where the judge prefers self_attested or a filled source_speaker;
    • 1 counterfactual polarity;
    • 1 rejected offer labelled a decision rather than a plan.

What the 34 remaining holds are:

  • the verifier calling the author of a plain-text note "user", which has no structure to ground;
  • 5 antecedents across different authors, left out on purpose;
  • name anchors on frames that do not contain the name, plus dates or common nouns listed as names. These are verifier mistakes.

Notes for the release

  • RETAIN_VERIFIER_VERSION v54 invalidates timeline-source extraction proofs, so timeline sources re-verify on upgrade. This matches earlier contract changes.
  • The cross-author case is the obvious next extension. The assistant mentions a product and Ada says "it" in the following turn; that was 5 cases here. It needs a deliberate decision about fix: retain source context through immutable antecedent witnesses #226's caution and is not included.

Testing

  • pnpm check:repository, build, lint, knip, smoke and docs:doctor pass.
  • pnpm test passes: 5024 passed in the full run. Three ingest.test.ts cases timed out at 30 s under full-suite load, and that file passes 70/70 on its own.
  • format:check flags only packages/core/src/ops/context.ts, which is unchanged from main.

🤖 Generated with Claude Code

The context witness rejected two kinds of legitimate references as
context_witness_invalid, which held the candidate as discourse_uncertain:

- a speakerless assistant or user turn has no author label, so the
  verifier's first-person "assistant" reference could not be grounded;
- a name stated in one turn could not anchor a pronoun or short name in
  the same author's later turn.

An unnamed assistant or user item now supplies its role as its author
label, and an unambiguous antecedent may come from an earlier item with
the identical author label. Another author's item, a later item and an
item without an author label still cannot donate an identity; explicit
names must still occur in their own frame. The verifier contract
describes both, so the verifier version advances to v54.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tenphi
tenphi merged commit 683c363 into main Oct 18, 2026
5 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant