Skip to content

feat: make relationship expansion admission-first and signal-aware (#190) - #191

Merged
ezutfen merged 3 commits into
mainfrom
feat/190-admission-first-relationship-expansion
Sep 7, 2026
Merged

feat: make relationship expansion admission-first and signal-aware (#190)#191
ezutfen merged 3 commits into
mainfrom
feat/190-admission-first-relationship-expansion

Conversation

@ezutfen

@ezutfen ezutfen commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

Closes #190 (ENG-RECALL-003D). Parent #160, epic #153.

What

Extends the governed/exploratory shadow recall profiles through the existing bounded graph/tunnel relationship expansion as an admission-first relevance mechanism: only V2-admitted direct candidates may seed discovery, every expanded neighbor is independently admitted through the exact risk_aware_shadow_v1 surface decision, and a versioned, importance-free relationship-relevance contract feeds the unchanged separated utility ranking.

A relationship can make a memory relevant. It can never make it trusted, epistemically supported, review-approved, or admissible.

Completion report

1. SHAs — start f62cf10 (main), head 359b2ac.

2. Files changedengram/relationship_recall.py (candidate discovery + pure relevance helper), engram/recall.py (admission-first wiring, origin-tagged diagnostics, merged V2 summaries), engram/recall_signals.py (live_proposal_expression moved here; signal_item_fields relevance param) , engram/recall_shadow.py + engram/api/routes/recall_shadow.py (expansion packet field), sdk/engram-client/tests/test_models.py (passthrough), tests/test_relationship_relevance.py (new, 11 unit tests), tests/test_recall_profile_semantic.py (24 new integration tests), docs/adr-160-recall-profiles.md (supplement).

3. Execution order (normative, mechanical) — direct semantic candidates → exact V2 admission on direct candidates → seeds chosen only from admitted direct candidates (admitted[:recall_semantic_expansion_seed_limit]; zero admitted seeds ⇒ no expansion) → bounded graph/tunnel discovery (relationship_recall.discover_candidate_neighbors) → exact V2 admission on every expanded neighbor through the same decide_recall_admission gate → relationship-aware relevance (relationship-relevance-v1) → separated utility ranking (compute_signal_rank_score) → unchanged budget packing.

4. V2 resolution strategy — direct window: one resolve_bulk_v2_decisions call (existing). Expanded neighbors: exactly one additional bounded bulk resolution for the newly discovered neighbor set per packet + one bulk load_admission_bindings. Packet v2_resolution merges the two summaries (counts/queries add; same policy identity). Example: 1 direct + 3 neighbors → resolved_count: 4, query_count: 14 (7+7).

5. Relevance contractrelationship_recall.compute_relationship_relevance, version relationship-relevance-v1: clamp01(max(direct_score, 0.70·semantic + 0.15·strongest_edge + 0.10·tunnel)) over the existing weights minus importance; per-edge weights clamped to [0,1]; an unlinked direct item's relevance is exactly its similarity (pre-#190 values); links never demote a direct hit. Excluded inputs are structurally absent from the signature (unit-pinned).

6. Representative governed packet (seed similarity 1.0, derived_from 0.9 edge, ops-runbooks tunnel):

item origins relevance components admission
direct seed semantic (no block) 1.0 current + allow
graph neighbor ["graph"] 0.835 semantic 0.7 + graph 0.135 current + allow (own binding)
tunnel neighbor ["tunnel"] 0.8 semantic 0.7 + tunnel 0.10 current + allow (own binding)

7. Relevance cannot modify evidence/admission — relevance is computed after the admission decision, from four inputs the decision never reads back; the relationship block is additive. Mechanical tests: weight-1.0 supports edge leaves epistemic unknown; edge strength changes relevance only; importance changes utility/rank only; trust/confidence/exposure counters change nothing.

8. Withheld expanded-neighbor diagnostic — content-free entry with origin: "graph", reason_codes: ["v2_surface_review_required"], v2_resolution_status: "current", v2_surface_decision: "review_required", gates_disagree: true, full v2 binding block.

9. Boundedness — no provider call beyond the one shared query embedding (counted: 1); no per-neighbor assessment/V2 query (resolve-call count == 2 per packet; merged query count constant as neighbors grow 2→6); graph/tunnel caps enforced mechanically; deterministic id tie-break.

10. Boundaries — cross-tenant neighbor, other-principal private neighbor, out-of-workspace neighbor, and unreadable middle-item inference all tested negative (no packet entry, no diagnostic, no id leak).

11. Legacy byte-compatexpand_recall_candidates, legacy tunnel fetcher (importance-first ordering), and the legacy packet shape are untouched; the full pre-existing legacy suite passes unchanged.

12. Shadow read-only — with expansion active: no recall log, no exposure counters, no review/promotion/assessment mutation (snapshot-equal).

13. Focused tests — 11 unit + 24 integration new; all 26 issue scenarios covered (scenario 19 via the unreadable-middle-item test; 24 via the pre-existing legacy suites + shape pin).

14. make check — lint + strict mypy green; root pytest failures are environment-only and proven identical to (a subset of) current main at f62cf10 (baseline reproduces every failure, plus more).

15. make compose-ci — ✅ at exact head 359b2ac: 3895 passed, 35 skipped — "All full CI checks passed" (migrations, app-role FORCE RLS, root + SDK + MCP + hooks suites against real PostgreSQL/pgvector).

16. GitHub CI — will attach the exact-head run below.

17. Production boundaryCERTIFIED_SERVING_PROFILES == {"legacy"} unchanged; /v1/recall still 422s on governed/exploratory; no MCP profile selection; no #161 corroboration, no Context Ledger semantic receipts, no #162 certification/cutover.

18. Remaining #160 slices — conflict-preserving/diversity packing over direct+expanded candidates; demonstrated-usefulness feedback in utility; semantic Context Ledger receipts; review/historical recall surfaces; dogfood evaluation + exposure-concentration analysis (now measuring expansion contribution); #162 certification + default cutover. (#160 progress comment follows after merge.)

ADR-160 supplement added: candidate relationship expansion is admission-first; only admitted direct items seed; every expanded neighbor gets its own exact V2 decision; graph/tunnel contribution is relevance, never evidence; importance is utility, not relationship relevance; legacy scoring is compatibility-only.

)

Extend the governed/exploratory shadow recall profiles through the bounded
graph/tunnel relationship expansion as an admission-first relevance
mechanism: only V2-admitted direct candidates may seed discovery, every
expanded neighbor is independently admitted through the exact
risk_aware_shadow_v1 surface decision, and a versioned importance-free
relationship-relevance contract (relationship-relevance-v1) feeds the
separated utility ranking. Legacy expansion remains byte-compatible;
production stays legacy-only.
#190)

Two review corrections to the admission-first relationship expansion,
both preserving the #190 architecture:

Graph: already-evaluated direct candidates (including V2-withheld ones)
no longer consume max_graph_neighbors_per_item/max_graph_expanded_items
capacity. They receive origin-merging enrichment links at zero capacity
cost; only genuinely new neighbor ids compete for the bounded windows,
so a withheld direct hit can never suppress a new expansion candidate.

Tunnel: admitted direct items in tunnel-linked (wing, room)s now collect
tunnel-origin metadata (semantic+tunnel / semantic+graph+tunnel origins),
via per-target enrichment queries with the identical eligibility
predicates — no second V2 admission, no new-neighbor budget consumption,
no broadened visibility.

Expansion accounting (graph_neighbors / tunnel_neighbors /
discovered_neighbors / admitted_expanded / withheld_expanded) now counts
genuinely new expansion candidates only. Legacy expand_recall_candidates
and the legacy packet stay byte-identical.
…d each neighbor (#190)

Tunnel links on the candidate-profile path now carry the admitted seed
whose tunnel membership exposed the (wing, room) a neighbor was pulled
from, mirroring _GraphLink.seed_id: _tunnel_targets accumulates every
seed that reaches each target, _fetch_candidate_tunnel_neighbors records
one deduped link per reaching seed (budget still counts genuinely new
neighbors only), and _candidate_relevance() derives source_seed_score
from the similarities of the seeds that actually reached the item —
never the packet-level best seed. Legacy expansion is unchanged.
@ezutfen
ezutfen merged commit c1d5f24 into main Sep 7, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ENG-RECALL-003D — Make relationship expansion admission-first and signal-aware

1 participant