Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.d/local-scoring-owner-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Local scoring ownership contract

- Identified active Python lineage-channel, reconstruction-decision, and
corporate-entity similarity paths as migration debt.
- Defined the fail-closed owner envelopes required before those paths can be
removed, without assigning corporate identity or moving existing heuristics
to another repository by assumption.
11 changes: 7 additions & 4 deletions docs/adr/0208-externalize-local-mathematical-compute.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
**Amends:** ADR 0003, ADR 0024, ADR 0064, ADR 0084, ADR 0132, ADR 0145,
ADR 0148, ADR 0167, ADR 0168, ADR 0182, ADR 0185, ADR 0200, ADR 0201, and
ADR 0205
**Extended by:** [ADR 0245](0245-lineage-scoring-and-entity-resolution-owner-contract.md),
which names the remaining channel, reconstruction-decision, and corporate-
entity similarity paths and defines their minimum owner envelopes.

## Context

Expand All @@ -28,9 +31,10 @@ The ecosystem product boundaries are already sufficient:
CPU/GPU implementation before LineageWeave treats a new result as governed
numerical evidence.

LineageWeave has no standalone canonical PRD file on this exact head. Until
one lands, `ARCHITECTURE.md` and the accepted ADR set are the product baseline;
this absence remains a product-documentation gap, not permission to infer a
At the time of this decision LineageWeave had no standalone canonical PRD.
`docs/product-requirements.md` has since landed as a supporting product
contract and confirms the same consumer-only measurement boundary; accepted
ADRs remain normative. The earlier absence was never permission to infer a
different responsibility.

## Decision
Expand Down Expand Up @@ -113,4 +117,3 @@ https://doi.org/10.1007/s11336-021-09762-5
Roberts, M. E., Stewart, B. M., & Tingley, D. (2019). stm: An R package for
structural topic models. *Journal of Statistical Software, 91*(2), 1–40.
https://doi.org/10.18637/jss.v091.i02

105 changes: 105 additions & 0 deletions docs/adr/0245-lineage-scoring-and-entity-resolution-owner-contract.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# ADR 0245 — Lineage scoring and entity resolution require owner artifacts

**Decision status:** Accepted
**Date:** 2026-08-26
**Amends:** ADR 0026, ADR 0064, ADR 0084, and ADR 0208

## Context

ADR 0208 freezes local numerical computation as migration debt, but its audit
does not name every active path. Protected `main` still executes the following
Python decisions during ordinary reconstruction and ingestion:

- `lineageweave/channels.py` computes an inverse elapsed-day score, a numeric
secondary-key score, and `difflib.SequenceMatcher` text similarity;
- `lineageweave/reconstruct.py` renormalizes weights, limits candidates to the
latest 50 records, applies a fixed `0.3` score floor, and invokes weighted
fusion before selecting a parent; and
- `lineageweave/corporate_hierarchy_resolution.py` deletes a fixed suffix
vocabulary, computes `SequenceMatcher` similarity, and applies a fixed `0.6`
catalog-binding threshold.
Comment thread
seonghobae marked this conversation as resolved.

Those paths affect product facts or evidence selection. They are not merely
display formatting or operational resource accounting. The cited record-
linkage literature supports an explicit uncertain outcome, but it does not
validate these particular constants or Python string-similarity rules.

Current ecosystem contracts do not provide a complete replacement. TEPP's
published LineageWeave project-history exchange returns temporal association,
not candidate-parent scores. RankWeave owns fusion and retrieval, but its
current public contract does not return a Rust-computed Event-Lineage edge
artifact or organization-identity decision. contextual-orchestrator owns
embedding transport and model orchestration, not catalog identity. Keyverse
owns account identity and is not implicitly assigned corporate-master entity
resolution.

Open PR #704 at audited head `ea6c5c8e9819590dfbc058344435122584947f6e`
publishes a useful external evidence envelope, but its analysis implementation
imports `_best_parent` and `active_weights`, computes candidate-window counts
and per-channel contributions, and applies caller policy score floors. It is
therefore a consumer-contract delivery, not the owner-compute replacement
required by this decision; its local arithmetic must not be cited as closing
this gap.

## Decision

1. **No local scoring extension.** The three named modules are frozen migration
debt. No new decay, normalization, token overlap, similarity algorithm,
candidate-order rule, score floor, threshold, or numeric fallback may be
added. Tests may characterize legacy behavior but may not call it calibrated,
paper-grounded, or release-compliant.
2. **Event-Lineage owner envelope.** Replacement activates only from a
versioned owner result containing:
- contract and result-schema versions;
- immutable input snapshot digest and knowledge cutoff;
- every considered record id and the evidence-unit references authorized by
LineageWeave before submission;
- selected parent id or an explicit abstention;
- separate temporal, grouping, semantic, and adjudication evidence with
availability status;
- owner model/method version, convergence or completion status, uncertainty
where the method defines it, and deterministic result digest; and
- an explicit non-causal classification.
TEPP owns calibrated temporal/event criterion evidence. RankWeave may own the
Rust-backed candidate ranking/fusion artifact after its PRD and API accept
that responsibility. LineageWeave validates and persists the envelope; it
never recomputes, renormalizes, thresholds, or repairs it.
3. **Corporate-entity resolution envelope.** No existing repository is assigned
this construct by inference. An owning repository must first accept a PRD/ADR
and publish a versioned result containing the input snapshot digest, bounded
candidate catalog ids, source/alias evidence references, `unique`/`miss`/`tie`
outcome, selected catalog id only for `unique`, method/model version,
uncertainty or review status, and result digest. Until then, no new automatic
similarity binding path may activate.
4. **Resource limits are not evidence.** A bounded request may cap records or
bytes before an owner call, but recency, truncation order, or a fixed window
must not decide scientific relevance. The request records any truncation and
the result remains incomplete rather than silently treating excluded
candidates as negative evidence.
5. **Deletion sequence.** After exact owner contracts land and pass synthetic
recovery/equivalence tests, separate consumer PRs add strict adapters and
persisted provenance. Only then do deletion PRs remove the corresponding
functions and constants. Missing, malformed, stale-digest, incomplete, or
unavailable owner results produce no edge or catalog binding.

## Consequences

- Existing runtime behavior remains honestly labeled migration debt rather
than being moved unchanged to another repository.
- LineageWeave cannot claim the affected reconstruction or automatic catalog
binding paths satisfy its product boundary until their local implementation
is deleted.
- Exact equality, sorting, authorization, persistence, and UI projection may
remain local when they do not manufacture a score or identity decision.
- The customer sees an unavailable or review-needed state instead of a guessed
lineage edge or organization identity when the owner artifact is absent.

## References — APA 7th

Bhattacharya, I., & Getoor, L. (2007). Collective entity resolution in
relational data. *ACM Transactions on Knowledge Discovery from Data, 1*(1),
Article 5. https://doi.org/10.1145/1217299.1217304

Fellegi, I. P., & Sunter, A. B. (1969). A theory for record linkage. *Journal
of the American Statistical Association, 64*(328), 1183–1210.
https://doi.org/10.2307/2286061
2 changes: 1 addition & 1 deletion docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ decision from them.
| [`operability/mcp-concurrency-evidence.md`](../operability/mcp-concurrency-evidence.md) | [0218](0218-current-contract-mcp-global-ask.md) |
| Evidence operations Dashboard (`/`) | [0206](0206-evidence-operations-dashboard.md) |
| [`temporal-topic-context-influence-research.md`](../temporal-topic-context-influence-research.md) | [0210](0210-temporal-topic-context-influence-dashboard.md) |
| [`python-mathematical-compute-boundary-audit.md`](../doctoring/python-mathematical-compute-boundary-audit.md) | [0208](0208-externalize-local-mathematical-compute.md) |
| [`python-mathematical-compute-boundary-audit.md`](../doctoring/python-mathematical-compute-boundary-audit.md) | [0208](0208-externalize-local-mathematical-compute.md), [0245](0245-lineage-scoring-and-entity-resolution-owner-contract.md) |

[0011](0011-prov-o-standard-relations.md) and [0065](0065-prov-o-provenance-boundary.md) cite the dated W3C PROV-O and PROV-DM Recommendations (https://www.w3.org/TR/2013/REC-prov-o-20130430/ and https://www.w3.org/TR/2013/REC-prov-dm-20130430/).

Expand Down
12 changes: 10 additions & 2 deletions docs/doctoring/python-mathematical-compute-boundary-audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,9 @@ Python paths satisfy the Rust/GPU requirement.

## Product-boundary sources read

- LineageWeave `ARCHITECTURE.md` and accepted ADRs 0003, 0132, 0145,
0200, 0201, and 0205. This exact head has no standalone canonical PRD.
- LineageWeave `docs/product-requirements.md`, `ARCHITECTURE.md`, and accepted
ADRs 0003, 0132, 0145, 0200, 0201, 0205, 0208, and 0245. The PRD is a
supporting product contract and ADRs remain normative.
- TEPP `docs/product/prd-v0.4-approved.md`, whose approved TRSL-TM scope
owns temporal, relational, multilingual, topic, event, and trajectory
measurement.
Expand All @@ -22,12 +23,14 @@ Python paths satisfy the Rust/GPU requirement.

| Current LineageWeave path | Local computation | Owner | Consumer replacement | Principal callers / tests |
|---|---|---|---|---|
| `lineageweave/channels.py` | inverse elapsed-day scoring, numeric secondary-key scoring, and `SequenceMatcher` label similarity | TEPP for calibrated temporal/event criterion evidence; RankWeave for Rust-backed similarity/ranking only after an accepted owner API | ADR 0245 Event-Lineage owner envelope; no local score fallback | `lineageweave/reconstruct.py`, estimation script; `tests/test_channels.py` |
| `lineageweave/channel_weight_estimation.py` | dichotomization, synthetic simulation, MLS2PLM input construction, expected item information and normalization | fast-mlsirm, conditional on TEPP anchor | versioned anchored-weight artifact; strict digest/convergence validation | estimation scripts, seed/server/rebuild paths; `tests/test_channel_weight_estimation.py`, estimator-script tests |
| `lineageweave/period_report.py` | response matrix, GRM/GPCM fit/FIPC/EAP, likelihood, category expectation, information ordering | fast-mlsirm | period-measurement artifact with item bank, scores, uncertainty, diagnostics | report ingestion and demo seed; period-report and report API tests |
| `lineageweave/leftover_pairs.py` | residual matrix, complete-case selection, SVD/Gabriel coordinates, distances, reconstruction, axis shares | fast-mlsirm | residual-interaction artifact with observed/expected identity and coverage | `period_report.py`, report ingestion/seed; `tests/test_leftover_pairs.py`, report tests |
| `lineageweave/embedding_client.py` and `backend/app/post_chat_ingestion.py` | cosine similarity, vector norms, maximum semantic score | RankWeave retrieval-score contract | ranked evidence envelope over ABAC-visible semantic units | reconstruction text channel and Global Ask retrieval; embedding/post-chat tests |
| `lineageweave/knowledge_graph.py` | random walk with restart, convergence delta, adaptive relevance cutoff | RankWeave graph-ranking contract | ranked-node artifact with contribution and convergence evidence | related-person/entity API paths; knowledge-graph tests |
| `lineageweave/reconstruct.py` | channel-weight renormalization, candidate-score fusion and minimum-score decision | RankWeave fusion; TEPP supplies independent lineage criterion | accepted edge-ranking artifact; LineageWeave persists selected edge and channel provenance | lineage rebuild/start/seed/server; reconstruct, persistence, API tests |
| `lineageweave/corporate_hierarchy_resolution.py` | fixed legal-suffix deletion, `SequenceMatcher` candidate score, and fixed catalog-binding threshold | unassigned; no ecosystem PRD/API currently accepts corporate-master entity resolution | ADR 0245 corporate-entity resolution envelope; remain unbound when unavailable | corporate/team/Keyman/entity-relationship ingestion; corporate resolution and tie tests |
| `lineageweave/rankweave_client.py` | channel construction, token overlap, RRF weights and contribution arithmetic | RankWeave | strict ranking artifact exposing owner-computed contributions | `/api/rankings`, frontend Rankings; `tests/test_rankweave_client.py` and frontend tests |

`lineageweave/post_evaluation.py` imports fast-mlsirm only for its published
Expand All @@ -51,6 +54,11 @@ validation, and presentation formatting also remain LineageWeave concerns.
- **RankWeave:** Rust-backed similarity, graph ranking, fusion, contribution,
evaluation, and policy-selection artifacts. Its present Python calculation
core is the correct product owner but not the final execution architecture.
- **Corporate entity resolution:** no owner is designated. A repository must
explicitly accept this product responsibility and ADR 0245's versioned
unique/miss/tie envelope before LineageWeave can replace the current local
candidate scorer. Neither contextual-orchestrator nor Keyverse acquires this
responsibility implicitly.

## Persistence and UI blast radius

Expand Down
1 change: 1 addition & 0 deletions docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -469,6 +469,7 @@ this file per §3.5 of the prior snapshot).

| Gap | Current evidence | Acceptance requirement |
| --- | --- | --- |
| Local lineage and organization scoring ownership | Protected `main` still executes Python elapsed-day decay, numeric secondary-key scoring, `SequenceMatcher` label/entity similarity, weight renormalization, a 50-record recency window, and fixed `0.3`/`0.6` decision floors. ADR 0208 froze this class of work, but no open PR supplies the complete owner artifacts and no repository currently accepts corporate-master entity resolution. TEPP project-history is temporal association only; current RankWeave APIs do not return the ADR 0245 Rust-computed edge or entity-resolution envelopes. Open PR #704 exact head `2948812e` adds a useful external evidence contract but reuses `_best_parent`/`active_weights` and locally computes window counts and contribution arithmetic, so it is not replacement-owner evidence | Land the versioned owner contracts in the construct-owning repositories, including snapshot/cutoff, separate evidence availability, method/model version, uncertainty/completion, digest, abstention/tie, and non-causal status. Then add strict LineageWeave adapters and persisted provenance before deleting `channels.py` scoring, local reconstruction normalization/window/floor decisions, and corporate `SequenceMatcher`/threshold binding. Missing owner evidence must produce no edge or catalog identity; do not transfer the existing heuristics upstream |
| Protected release | 17 open PRs at snapshot. Sixteen target `main` with normal auto-merge enabled; stacked child #703 targets #640 and must wait for its parent, then retarget to `main` and collect fresh evidence. None has the required independent approval, and queued checks are not treated as blockers for safe work on other PRs | Terminal exact-head checks, no unresolved threads, the current ruleset's one independent approval, and protected squash-merge SHA |
| CI queue release latency | Two Tests runs for already merged PRs occupied the available runner slots while 54 newer runs remained queued. Manual cancellation released the stale work, but the central close workflow was itself queued behind those runs. #634 merged into #631's non-default branch and reuses the repository's existing per-PR concurrency group so a jobless close event can cancel obsolete Tests work before runner allocation; this is not protected-main delivery | Merge #631 through its refreshed protected gate; close a synthetic PR while its Tests run is active and verify the old run becomes cancelled, the close-event jobs remain skipped, and a newer exact-head run starts without manual intervention |
| Evidence-grounded operations workspace | Protected-main #614 delivers governed semantic Ask, live Similar VOC, disjoint pending/failed analysis metrics, full Storybook state inventory, and current desktop/mobile screenshot evidence. Authorized-corpus backfill acceptance remains unavailable | Perform authenticated authorized-corpus acceptance with aggregate evidence and retain fail-closed no-match behavior |
Expand Down