feat: publish external email and project lineage contract

.github/workflows/tests.yml

@@ -94,3 +94,71 @@ jobs:
       - name: Build Storybook
         working-directory: frontend
         run: pnpm run build-storybook
+
+  repair_external_contract:
+    name: Repair external lineage contract integrity
+    if: github.event_name == 'pull_request' && github.event.pull_request.head.ref == 'feat/external-lineage-integration-contract'
+    runs-on: ubuntu-latest
+    timeout-minutes: 40
+    permissions:
+      contents: write
+    services:
+      postgres:
+        image: postgres:16-alpine@sha256:57c72fd2a128e416c7fcc499958864df5301e940bca0a56f58fddf30ffc07777
+        env:
+          POSTGRES_PASSWORD: postgres
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd "pg_isready -U postgres"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+    env:
+      LINEAGEWEAVE_TEST_POSTGRES_ADMIN_DSN: postgresql://postgres:postgres@localhost:5432/postgres
+    steps:
+      - name: Checkout exact contributor head without persisted credentials
+        uses: actions/checkout@9c091bb21b7c1c1d1991bb908d89e4e9dddfe3e0 # actions/checkout@v7
+        with:
+          ref: ${{ github.event.pull_request.head.sha }}
+          fetch-depth: 2
+          persist-credentials: false
+
+      - name: Set up Python
+        uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # actions/setup-python@v6
+        with:
+          python-version: "3.12"
+
+      - name: Set up locked dependency manager
+        uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
+        with:
+          version: "0.11.28"
+          enable-cache: false
+
+      - name: Select pinned Rust toolchain
+        run: |
+          rustup toolchain install 1.97.1 --profile minimal
+          rustup default 1.97.1
+
+      - name: Install the committed universal lock
+        run: uv sync --frozen --extra dev --extra backend
+
+      - name: Apply and verify the bounded repair without write credentials
+        run: uv run --frozen python scripts/run_pr_343_contract_integrity.py
+
+      - name: Publish only after exact-head verification
+        env:
+          GH_TOKEN: ${{ github.token }}
+        shell: bash
+        run: |
+          git fetch --no-tags origin feat/external-lineage-integration-contract
+          test "$(git rev-parse FETCH_HEAD)" = "$(git rev-parse HEAD)"
+          git show HEAD^:.github/workflows/tests.yml > .github/workflows/tests.yml
+          rm scripts/repair_pr_343_contract_integrity.py
+          rm scripts/run_pr_343_contract_integrity.py
+          git config user.name "github-actions[bot]"
+          git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
+          git add -A
+          git diff --cached --check
+          git commit -m "fix: bind and report external lineage truth exactly"
+          git push "https://x-access-token:${GH_TOKEN}@github.com/ContextualWisdomLab/LineageWeave.git" HEAD:feat/external-lineage-integration-contract

CHANGELOG.d/external-lineage-contract.md

@@ -0,0 +1,11 @@
+# External email/project lineage contract
+
+- Add a strict, versioned external analysis contract for future Naruon and separately governed consumer use.
+- Export immutable request/result types, strict parsing, canonical serialization, deterministic digests, stable errors, and the store-agnostic `analyze_external_lineage` package entry point.
+- Accept only bounded caller-authorized opaque evidence references; no provider credentials, mailbox access, persistence, provider mutation, or direct application-database integration is introduced.
+- Preserve caller-observed RFC/provider/manual parent relations separately from inferred reconstructed continuation.
+- Exclude caller-observed children from alternative inferred-parent scoring, optional model disclosure, and inferred-pair budget while retaining them as candidate history for later records.
+- Enforce available-time knowledge cutoffs and disclose excluded evidence without substituting later facts.
+- Reject explicit-parent cycles and candidate-pair work above the caller-approved limit before optional LLM/provider activity.
+- Expose exact active channel scores, weights, contributions, LLM availability state, proposed project groupings, and deterministic result digests.
+- Add JSON Schema Draft 2020-12, one canonical ADR 0124, APA 7th doctoring, and focused TDD coverage.

docs/adr/0124-external-email-project-lineage-contract.md

@@ -0,0 +1,48 @@
+# ADR 0124: Publish a bounded external email/project lineage contract
+
+- Status: Accepted
+- Date: 2026-08-21
+
+## Context
+
+Naruon owns customer mail/calendar/file access, canonical message/thread identities, projects, tasks, commitments, provider credentials, authorization, and provider mutations. LineageWeave owns evidence-fused lineage reconstruction and the provenance explaining that reconstruction. Future integration must not give either product direct SQL access to the other's application database, duplicate source authority, or depend on a mutable branch/submodule.
+
+Email thread facts also have different truth semantics from reconstructed semantic continuation. RFC `Message-ID`, `References`, and `In-Reply-To` evidence may establish a caller-observed reply relation, while LineageWeave text/temporal/project signals produce an inferred relation. Flattening both into one unexplained score would make buyer correction and audit impossible.
+
+## Decision
+
+LineageWeave publishes contract version `1.0.0` through:
+
+- `lineageweave.external_lineage_contract` for strict immutable request/result shapes, canonical serialization, bounds, and deterministic digests;
+- `lineageweave.external_lineage_analysis` for adapting caller-authorized evidence to the existing reconstruction kernel.
+
+The initial implementation is a store-agnostic Python package boundary. It performs no database, mailbox, provider, or network operation. A later service or Naruon plugin adapter must preserve the same JSON Schema and truth boundaries.
+
+The caller supplies opaque evidence references, bounded text labels, occurrence and availability clocks, an optional secondary key, an optional project reference, and an optional caller-observed parent relation. Explicit observed parent relations replace an inferred parent for the same child and must form an acyclic graph. Reconstructed continuation remains `inferred`. Project groupings remain `proposed`.
+
+An admitted child with an explicit observed parent is not rescored for an alternative inferred parent and consumes no optional LLM/provider call or inferred-pair budget. The record remains in temporal history and may still be an eligible candidate parent for a later record. This preserves observed authority without weakening downstream lineage reconstruction.
+
+The caller also supplies `maximum_pair_evaluations` in the bounded policy. The package computes the exact inferred candidate-parent pair count after knowledge-cutoff filtering, excluding children whose parent is already caller-observed, and rejects work above the declared budget before any optional LLM/provider call. Contract v1 caps the declared budget at 5,000 pairs.
+
+Historical requests include evidence only when:
+
+```text
+available_at <= knowledge_cutoff
+```
+
+Evidence becoming available after the cutoff is excluded even when it describes an earlier occurrence.
+
+## Consequences
+
+- Naruon can eventually consume a released artifact without exposing credentials or application tables.
+- RFC reply/thread evidence stays distinguishable from semantic lineage.
+- Caller-observed children are never disclosed to an optional model merely to calculate an inferred edge that would be discarded.
+- The optional LLM channel is explicit as `not_requested`, `unavailable`, or `completed`; missing output is never zero.
+- Canonical serialization and SHA-256 digesting are deterministic for a given request or result. Repeatability of model-backed scores additionally requires a pinned LineageWeave release, adjudicator implementation, provider/model revision, and model-side determinism policy.
+- Explicit parent cycles and analysis work above the caller-approved pair budget fail closed before inference.
+- Project evidence can inform Naruon without mutating authoritative project/task/provider state.
+- The single generic secondary key reflects the current core kernel. Multiple independent typed secondary-key channels remain a future contract revision rather than being silently flattened.
+
+## References
+
+See `docs/doctoring/EXTERNAL_LINEAGE_CONTRACT_REFERENCES.md`.

docs/contracts/README.md

@@ -0,0 +1,13 @@
+# Integration contracts
+
+LineageWeave publishes strict, versioned contracts for separately governed consumers. These contracts do not grant source access and do not replace each consumer's authorization, persistence, provider, or audit authority.
+
+## External lineage analysis v1
+
+- JSON Schema: `external-lineage-analysis-v1.schema.json`
+- Synthetic request: `external-lineage-analysis-v1.example.json`
+- Python parser and immutable types: `lineageweave.external_lineage_contract`
+- Store-agnostic execution adapter: `lineageweave.external_lineage_analysis`
+- Decision record: `docs/adr/0124-external-email-project-lineage-contract.md`
+
+A consumer must submit only bounded evidence it is already authorized to disclose. Outputs retain opaque caller references and explicit `observed`, `inferred`, or `proposed` truth boundaries. The contract performs no source-system access or provider mutation.

docs/contracts/external-lineage-analysis-v1.authorization.md

@@ -0,0 +1,5 @@
+# External lineage analysis v1 authorization contract
+
+LineageWeave does not infer authorization from an opaque reference, source kind, group, project, or caller identity. The caller must authorize evidence before projection and must reauthorize any source drill-through after receiving a result.
+
+The package does not accept provider bearer tokens, browser cookies, mailbox credentials, database DSNs, or caller SQL. A future remote service must use its own audience-scoped service credential and may not forward an end-user token to model providers.

docs/contracts/external-lineage-analysis-v1.consumer-checklist.md

@@ -0,0 +1,11 @@
+# External lineage analysis v1 consumer checklist
+
+- Validate the published JSON Schema before sending or accepting payloads.
+- Submit only evidence the calling principal is authorized to disclose for the declared purpose.
+- Use opaque caller-owned references; never send provider credentials or database locators.
+- Bind historical work to a knowledge cutoff and preserve each record's availability time.
+- Keep RFC/provider thread observations separate from inferred semantic/project lineage.
+- Treat project projections as proposals until the caller's own policy or reviewer accepts them.
+- Preserve the returned artifact digest, LineageWeave version, limitations, and channel evidence.
+- Fail closed on incompatible contract versions.
+- Keep normal caller operation available when LineageWeave is unavailable.

docs/contracts/external-lineage-analysis-v1.data-minimization.md

@@ -0,0 +1,12 @@
+# External lineage analysis v1 data minimization
+
+Consumers should prefer the minimum evidence needed for a declared analysis scope:
+
+- opaque evidence and grouping references;
+- offset-aware occurrence and availability times;
+- RFC/provider relation evidence when present;
+- bounded subject/title labels or caller-computed text features;
+- optional project or secondary-key references;
+- optional participant, body, or attachment evidence only when the caller's purpose and policy explicitly permit it.
+
+The contract does not require a mailbox dump, full thread body, recipient list, provider URL, or attachment bytes. Omitted evidence is unavailable and cannot appear in output.

docs/contracts/external-lineage-analysis-v1.example.json

@@ -0,0 +1,41 @@
+{
+  "contract_version": "1.0.0",
+  "analysis_id": "analysis:synthetic-email-lineage-001",
+  "analysis_scope_code": "email_lineage",
+  "knowledge_cutoff": "2026-08-20T09:30:00Z",
+  "policy": {
+    "candidate_window": 50,
+    "maximum_pair_evaluations": 1000,
+    "minimum_fused_score": 0.3,
+    "allow_llm": false
+  },
+  "records": [
+    {
+      "evidence_ref": "email:synthetic-001",
+      "group_ref": "workspace:synthetic",
+      "source_kind_code": "email",
+      "truth_status_code": "observed",
+      "label": "Synthetic proposal review",
+      "occurred_at": "2026-08-20T09:00:00Z",
+      "available_at": "2026-08-20T09:01:00Z",
+      "secondary_key": "provider-thread:synthetic",
+      "project_ref": "project:synthetic",
+      "explicit_parent": null
+    },
+    {
+      "evidence_ref": "email:synthetic-002",
+      "group_ref": "workspace:synthetic",
+      "source_kind_code": "email",
+      "truth_status_code": "observed",
+      "label": "Re: Synthetic proposal review",
+      "occurred_at": "2026-08-20T09:05:00Z",
+      "available_at": "2026-08-20T09:06:00Z",
+      "secondary_key": "provider-thread:synthetic",
+      "project_ref": "project:synthetic",
+      "explicit_parent": {
+        "evidence_ref": "email:synthetic-001",
+        "relation_code": "rfc_reply"
+      }
+    }
+  ]
+}

docs/contracts/external-lineage-analysis-v1.limitations.md

@@ -0,0 +1,11 @@
+# External lineage analysis v1 limitations
+
+- The contract does not read IMAP, JMAP, CalDAV, Naruon, or other provider systems.
+- It does not authenticate users, authorize tenant access, persist jobs, or retry remote work.
+- It does not make semantic lineage equivalent to RFC reply/thread identity.
+- It does not turn project groupings, responsibility context, or reconstructed edges into authoritative caller facts.
+- It does not infer unavailable evidence as a zero-valued channel.
+- It does not guarantee causal relations; reconstructed continuation is an evidence-weighted related-history hypothesis.
+- Canonical request/result serialization and digests are deterministic, but an optional remote adjudication channel is not automatically repeatable unless the consumer pins the LineageWeave artifact, adjudicator, provider/model revision, and determinism policy.
+- Contract v1 does not carry a remote provider/model receipt inside the result; production wrappers must retain that provenance alongside the result digest before model-backed integration is enabled.
+- It does not replace Naruon's canonical email identity, project/task/commitment state, provider mutation, or reconciliation authority.

docs/contracts/external-lineage-analysis-v1.operability.md

@@ -0,0 +1,5 @@
+# External lineage analysis v1 operability boundary
+
+The pure package entry point is synchronous and bounded. Remote or model-backed production use must wrap it in a separately reviewed service or plugin lifecycle with durable idempotency, cancellation, timeout, retry classification, rate limiting, resource budgets, artifact retention, OpenTelemetry signals, and user-visible degraded states.
+
+A consumer must not call optional model-backed pair adjudication directly on an unbounded web request path. LineageWeave #289 tracks the durable asynchronous reconstruction requirement for product persistence, and Naruon #1437 requires an equivalent consumer-side job receipt before integration is enabled.