📝 Decide the v0.1.6 embedded vector recall phase: service-free candidate recall on Qdrant Edge with a truthful completeness verdict and a five-field record

[ebigunso]

Docs only. Merging this PR approves the five decision records and the execution plan (the plan's status flips to approved in the first implementation PR).

Contents

• ADR-I-0023: the embedded vector candidate store is the in-process build of the service engine (Qdrant Edge), chosen on two feasibility spikes for a memory expected to span years to decades, and it is the default vector mode from this phase, licensed by the phase's own parity suite and service-free integration path (partially supersedes ADR-I-0003's vector default; the service adapter remains the explicit service mode); exactness is a threshold property; the in-house exact scan and sqlite-vec are rejected alternatives.
• ADR-I-0024: vector candidate recall reports a completeness verdict, and a vector-layer prefilter reads only synchronised or immutable values and never matches an unknown one; the scope-only query is recorded as current state, the type shape as a non-binding appendix.
• ADR-I-0025: the vector record is a five-field read contract; readable text lives in graph authority, the record keeps only the embedded surface as provenance, consumers hydrate by object id (partially supersedes ADR-I-0005 and ADR-I-0002's dual-text note).
• ADR-I-0026: raw vector baselines read the retrieval trace; no public candidate-search facade; cross-repository obligations for the evaluation repository.
• ADR-I-0027: the embedded engine runs on a dedicated blocking owner that flushes every write before acknowledging it (measured: the pinned engine persists only on flush and does not replay its log on load); signal-only facade drop; no new API.
• Phase document rewritten against the as-built port (the 2026-07-20 draft described capabilities the structured-verdict phase deleted); deletions promoted to deliverables; deferral-reconfirmation checklist; cross-repository section.
• Roadmap: v0.1.6 row, sequencing line, section 12 (later sections renumbered).
• Execution plan (draft): six library tasks in five waves, contract-first; evaluation-repository work is planned and tracked in that repository.
• Supersession note on the vector payload design note.

Basis

Planning wave 2026-09-02: a design consult, two altitude audits, and two forensic censuses, consolidated and ruled by the decider on the five open questions; decision records restructured 2026-09-03 on the decider's review (one decision per record, no withheld default, code shapes only in an appendix). The prerequisite evaluation-repository evidence-integrity fixes landed separately.

🤖 Generated with Claude Code

[Copilot]

🟡 Changes recommended

The documents contain conflicting default-mode, supersession, schema-version, and payload-shape claims.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Defines the v0.1.6 embedded vector recall architecture and implementation plan.

Changes:

• Adds four ADRs covering storage, recall completeness, vector records, and evaluation baselines.
• Updates the phase roadmap and supersession documentation.
• Adds an eight-task, five-wave execution plan.

File summaries

File	Description
docs/roadmap/development_roadmap.md	Adds and sequences v0.1.6.
docs/design/roadmap-phases/v0_1_6_embedded_vector_candidate_recall.md	Defines phase scope and acceptance criteria.
docs/design/database/vector_payload_design.md	Adds payload-design supersession guidance.
docs/decisions/implementation/ADR-I-0026-raw-vector-baselines-read-the-retrieval-trace.md	Moves baselines to retrieval traces.
docs/decisions/implementation/ADR-I-0025-vector-record-is-a-read-contract.md	Defines the five-field record contract.
docs/decisions/implementation/ADR-I-0024-vector-candidate-recall-reports-completeness-and-takes-a-scope-only-query.md	Defines recall verdicts and query scope.
docs/decisions/implementation/ADR-I-0023-embedded-sqlite-exact-scan-vector-candidate-store.md	Selects the embedded SQLite adapter.
docs/decisions/implementation/ADR-I-0005-qdrant-payload-vs-graph-authority.md	Records partial supersession.
docs/decisions/implementation/ADR-I-0003-qdrant-oxigraph-defaults.md	Records backend-decision supersession.
docs/decisions/implementation/ADR-I-0002-natural-language-embedding-surfaces.md	Records text-contract supersession.
docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md	Adds the phased execution plan.

Review details

Suppressed comments (1)

docs/roadmap/development_roadmap.md:1105

• This sequencing claim conflicts with the approved decision to keep service mode as the default. v0.1.6 enables a service-free local configuration and service-free default tests, but the default deployment still requires the service until the separately gated default flip.

v0.1.6 removes the last external-service dependency from the default deployment and test path

• Files reviewed: 11/11 changed files
• Comments generated: 6
• Review effort level: Balanced

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

🟡 Changes recommended

The proposed mixed-type baseline can produce incorrect per-kind results, and the embedded scoring contract omits query normalization.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:162

• Implementing mixed-type overfetch-and-slice can silently change the vector-only baseline on skewed score distributions: the global trace may contain fewer than a kind's budget even though more candidates of that kind exist below the global cutoff. Make Task_5 issue one traced retrieval per kind with a singleton scope (or implement real per-type limits) rather than relying on a multiplier and an A/B run over only the current fixtures.

  Replace the direct vector-service search in the vector-only baseline with the retrieval trace (overfetch-and-slice per kind; item text from the evaluation repository's own ingest records); mirror the completeness telemetry field; add the typed backend identity to result rows through the repository's clean-schema procedure; make the cleanup guard backend-neutral; drop the payload constants, the second vector client's search path, and the second embeddings client's divergent dimension handling. Perform the A/B run with a row-level diff of item identities and ranks against the pre-switch baseline before deleting the old path.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:273

• Exhaustive { scanned } is defined as “every stored record in scope was scored,” but the service adapter already short-circuits limit == 0 without scanning. Reporting an empty exhaustive result there would therefore be false whenever the scope contains records. Either add a truthful zero-limit verdict, reject zero at the boundary, or redefine the exhaustive contract and its scanned meaning consistently across both adapters.

- Edge cases: empty object-type scope selects zero in both adapters; `limit == 0` returns an empty exhaustive result; identical-vector tie fixtures must produce Exhaustive versus BoundaryTieClosed, never be encoded as expected parity of the bounded behavior.

• Files reviewed: 11/11 changed files
• Comments generated: 6
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The proposed contracts do not fully handle zero limits, zero-vector normalization, schema compatibility, or item-level trace deduplication.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 10/10 changed files
• Comments generated: 4
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

Several contract statements conflict, and the plan omits explicit ownership for zero-norm record validation.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (1) — in code that hasn't changed since the last review.

docs/design/roadmap-phases/v0_1_6_embedded_vector_candidate_recall.md:103

• This non-goal contradicts the phase's required service-adapter changes: the result envelope, five-field record, and payload-index deletion all alter that adapter. The roadmap already states the intended boundary as no changes beyond the shared contract; use that qualification here so implementers are not given mutually exclusive requirements.

docs/decisions/implementation/ADR-I-0026-raw-vector-baselines-read-the-retrieval-trace.md:51

• This obligation drops the surface multiplier required by the decision at lines 41–42 and by the phase/plan. A budget-only limit is insufficient once an object kind has multiple embedding surfaces, because duplicate surfaces can consume the top-K before the object-level budget is determinate. Keep the multiplier in the implementation obligation.

- The vector-only surface-policy validator keeps its object-type and budget rules; each measured kind becomes one singleton-scoped traced retrieval whose limit is that kind's section budget.

docs/decisions/implementation/ADR-I-0024-vector-candidate-recall-reports-completeness-and-takes-a-scope-only-query.md:126

• This description contradicts this ADR's own NotRequested variant for a zero limit (lines 45 and 52). The embedded adapter is exhaustive only when it performs a search; keeping “always” here can misstate the conformance requirement.

- ADR-I-0023 (the embedded adapter that always reports exhaustive completeness).

• Files reviewed: 10/10 changed files
• Comments generated: 2
• Review effort level: Balanced

[Copilot]

🔵 Needs a closer look

Contract inconsistencies and incorrect task ownership paths would leave the approved execution plan incomplete.

Review details

Suppressed comments (6)

Previously missed (3) — in code that hasn't changed since the last review.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:16

• This says every wave produces a PR in both repositories, but the wave rules later require a PR only for each touched repository. Waves 1, 3, and 5 do not schedule evaluation-repository changes, so the Definition of Done should use the same touched-repository condition.

This issue also appears in the following locations of the same file:

• line 45
• line 140
• line 206
  docs/decisions/implementation/ADR-I-0026-raw-vector-baselines-read-the-retrieval-trace.md:51

• This obligation drops the required surface multiplier, contradicting the Decision above and the phase/plan. If an object can have multiple surfaces, fetching only the section budget can omit an object whose best surface belongs in the top object budget even when the returned surface cohort is complete.
  docs/design/database/vector_payload_design.md:3
• The note says the whole Design Goal remains current, but rule 3 in that section still directs readers to treat relationship and lifecycle payload fields as hints. ADR-I-0025 removes those fields and explicitly supersedes that candidate-filter guidance, so the note currently leaves contradictory implementation advice marked as current.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:45

• This ownership glob does not match the repository's workflow files, which use the .yaml extension. Task 1 therefore cannot make its required CI live-gate change under the stated ownership.

  - .github/workflows/*.yml

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:206

• This ownership pattern matches no files, while retiring FakeVectorCandidateStore requires edits to inline test modules in src/memory.rs, src/usecases/correct_forget.rs, and src/usecases/retrieve.rs. Without owning those files, Task 7 cannot meet its zero-hit acceptance criterion.

  - src/**/tests (fake stores only)

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:140

• The phase lists a pipeline test over the embedded adapter with a deleted graph object as a deliverable, but no task schedules that integration coverage. The existing missing-object test uses RecordingVectorStore (src/usecases/retrieve.rs:2228), so the port parity suite alone will not verify the new adapter through graph-authority rejection.

  - Embedded mode constructs and retrieves with no service running; the parity suite yields identical admitted sets on the shared fixtures; the tie fixture yields Exhaustive (embedded) and BoundaryTieClosed (service).

• Files reviewed: 10/10 changed files
• Comments generated: 0 new
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The plan has repository-boundary conflicts and underspecified portability, runtime-blocking, documentation, and validation requirements.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (7)

Previously missed (5) — in code that hasn't changed since the last review.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:117

• clippy -D warnings is not a valid invocation and conflicts with the repository command documented in docs/coding-agent/rules/common.md:26. As written, Task_3's required validation cannot be executed.
  docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:140
• The phase deliverables require a pipeline test using the real embedded adapter with a deleted graph object, but Task_4 only accepts port parity. Without this test, the adapter-to-graph-authority integration can be omitted while the task still passes.

This issue also appears on line 141 of the same file.
docs/decisions/implementation/ADR-I-0023-embedded-sqlite-exact-scan-vector-candidate-store.md:46

• Using the public collection name directly as a filename breaks the promised backend-neutral namespace on case-insensitive filesystems (Foo and foo alias) and for reserved names such as CON. Character-set validation alone does not prevent these cases. Derive a collision-resistant filesystem-safe filename and record the original collection name in metadata.
  docs/design/roadmap-phases/v0_1_6_embedded_vector_candidate_recall.md:48
• The vector port is asynchronous, but this design carries the synchronous rusqlite mutex model into an exact scan that the document estimates may process about 120 MB per query. Running that work directly in search_candidates will block a Tokio worker and stall unrelated requests; require a blocking-pool or dedicated-worker boundary and include concurrent-request behavior in the benchmark.
  docs/design/roadmap-phases/v0_1_6_embedded_vector_candidate_recall.md:103
• This non-goal contradicts the same document's required service-adapter changes: the return envelope, completeness mapping, five-field payload, and dropped indexes all alter that adapter. Qualify the statement as the roadmap already does so implementers do not treat required contract work as out of scope.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:141

• The ADR requires reopen-time validation of vector size, distance, and supported record schema version, but this acceptance criterion only says a restart test passes. Make each incompatibility explicit so an implementation that validates only persistence or dimension cannot satisfy the task.

  - Restart test passes; repeated runs are byte-identical.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:206

• This wave list contains only library tasks after Tasks 5 and 6 moved to the evaluation repository, so claiming every wave is a PR pair is inaccurate and conflicts with independent planning. Describe rollback per library wave without promising synchronized cross-repository PRs.

- Each wave is a separately revertible PR pair.

• Files reviewed: 11/11 changed files
• Comments generated: 4
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The proposed decisions contain unresolved contract, supersession, path-safety, and execution-plan inconsistencies.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (3)

Previously missed (2) — in code that hasn't changed since the last review.

docs/decisions/implementation/ADR-I-0026-raw-vector-baselines-read-the-retrieval-trace.md:41

• This proof depends on a per-kind maximum-surfaces constant, but the current embedding-surface policy only provides one record builder per kind (src/policy/embedding_surface.rs:11-165), and no task adds a consumable constant. Since that module is crate-private and this ADR also forbids new public surface, the companion repository has no stable contract from which to choose the limit as surface generation evolves. Specify where this bound is exposed and add a drift-detection contract test, or use an adaptive retrieval rule that does not require the bound.
  docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:201
• The PR description says this is an eight-task execution plan, but the plan defines six tasks and explicitly confirms that Tasks 5 and 6 moved elsewhere. Update the PR description to say six tasks so the approved scope is unambiguous.

docs/decisions/implementation/ADR-I-0024-vector-candidate-recall-reports-completeness-and-takes-a-scope-only-query.md:45

• RetrievalTelemetry::default() currently preserves a configured vector limit of 48 (src/api/types/retrieval.rs:247-263,997-1001), so none of these variants can truthfully initialize its new required field: NotRequested specifically requires a zero limit, while the other variants require an adapter result. Add a non-result/default state (also suitable when deserializing older telemetry), or make the field optional while requiring the pipeline to populate it for completed retrievals.

    NotRequested,                                        // the limit was zero; no search was issued

• Files reviewed: 11/11 changed files
• Comments generated: 3
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The plan omits explicit path-confinement acceptance and does not address blocking synchronous exact scans on the async runtime.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

• Files reviewed: 12/12 changed files
• Comments generated: 3
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The configuration and filename contracts have unresolved portability gaps, and the execution plan contains incorrect or incomplete validation details.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (2)

Previously missed (2) — in code that hasn't changed since the last review.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:204

• The PR description says this execution plan has eight tasks, but the plan defines only Tasks 1–4, 7, and 8 (six tasks) and explicitly says 5 and 6 moved elsewhere. Align the description or the plan so the approved scope is unambiguous.
  docs/roadmap/development_roadmap.md:1105
• The newly inserted v0.1.6 step contradicts the preceding claim that v0.1.5 “closes the v0.1 family”; the v0.1.5 roadmap row repeats that claim. Update the closeout wording so the sequence does not declare the same release family closed before another v0.1.x phase.

• Files reviewed: 12/12 changed files
• Comments generated: 4
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The documents contain conflicting contract guidance and an incomplete async SQLite concurrency requirement.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (5)

Previously missed (4) — in code that hasn't changed since the last review.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:19

• The phase includes evaluation-repository outputs, and A2 explicitly says those outputs are produced under that repository's own plan, so “all in this repository” contradicts this plan's stated ownership. Scope this plan to the library deliverables and identify the external evidence as consumed rather than locally delivered.
  docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:205
• The PR description advertises an execution plan with eight tasks, but this file defines six tasks and confirms that identifiers 5 and 6 moved to another repository. Either update the PR description to say six library tasks or summarize the two external tasks here if eight is intended to describe the coordinated plan.
  docs/design/database/vector_payload_design.md:3
• The note says the whole Design Goal remains current, but its third rule still directs implementers to keep relationship and lifecycle payload hints. ADR-I-0025 removes those fields, and ADR-I-0024 prohibits bringing such predicates back without a synchronized/immutable re-entry path. Mark that rule as dated rationale too so this document does not retain conflicting implementation guidance.
  docs/roadmap/development_roadmap.md:262
• The “only the fields a reader consumes” claim omits the deliberate embedding_text provenance exception; the phase census says that field currently has no reader. Reflect the exception in the release row so it does not conflict with ADR-I-0025.

This issue also appears on line 1209 of the same file.

docs/roadmap/development_roadmap.md:1209

• This says every stored field is reader-consumed, but the same phase records that embedding_text currently has no reader and is retained as provenance. State that exception here so the roadmap matches ADR-I-0025's actual invariant.

Because a second adapter must implement the vector port, this phase also settles the port contract that the structured-verdict work deferred: recall completeness is reported instead of silently degraded, the query is scope-only, and the stored payload is exactly what a reader consumes.

• Files reviewed: 12/12 changed files
• Comments generated: 3
• Review effort level: Balanced

[Copilot]

🟡 Changes recommended

The prefilter contract has an unobservable failure path, and several records contain inconsistent dates, status, metadata, or authoring conventions.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Review details

Suppressed comments (4)

docs/decisions/implementation/ADR-I-0024-vector-candidate-recall-reports-completeness-and-prefilters-never-match-unknown.md:46

• The stated behavior does not make an incorrectly populated column a “visible failure”: an engine prefilter will simply omit records whose value is missing, creating the silent false negative that line 35 identifies as the more serious failure. Require an enable/open-time population check that fails explicitly (and add it to the implementation/validation plan), or remove the visibility claim and redesign how missing values are surfaced.

A vector-layer predicate may be evaluated only over a column that is fully populated for every searchable record (backfilled from graph authority before the predicate is enabled, per ADR-I-0025) and that is immutable or synchronised on every mutation; under those two conditions a missing or unknown value is a defect, not a state, and it never satisfies a positive predicate, so the rule produces no false negative on a correctly populated column and turns an incorrectly populated one into a visible failure rather than a silent widening.

docs/decisions/README.md:36

• This new rule is violated by the ADRs added in the same PR: examples include “same change” (ADR-I-0023:47), “already happened” (ADR-I-0024:9), “once” (ADR-I-0025:9), “at the time of this record” (ADR-I-0026:41), and “this phase” (ADR-I-0027:9). Rewrite those references using named versions, records, or absolute dates before establishing this as the repository rule.

- Records read the same at any time. No wording that hinges on when the record was written ("this phase", "later decision", "at decision time", "once", "already", change verbs such as "gains" or "moves"); history is anchored to named records, versions, and absolute dates, and Implementation Impact describes the resulting state.

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:224

• This cites a 2026-09-04 Decision Log entry, but the Decision Log ends with 2026-09-03; the merge-shape refinement is actually in the Progress Log below. Correct both the log reference and the future date.

Each wave ends with reviewer approval and a PR stacked on the previous wave's PR (GitHub stack on the planning PR); the next wave branches from the approved tip without waiting for a merge, and the decider merges the entire stack in one go at phase end (Decision Log, 2026-09-03 and 2026-09-04).

docs/coding-agent/plans/active/v0-1-6-embedded-vector-recall-plan.md:3

• The PR description says this plan's status flips to approved in the first implementation PR, but this docs-only planning diff already records it as approved. Keep this status pre-approval until the designated implementation change, or update the PR's stated approval workflow so the record and merge contract agree.

- status: approved

• Files reviewed: 17/17 changed files
• Comments generated: 1
• Review effort level: Balanced