Skip to content

✨ The vector record becomes a five-field read contract, and zero-norm embeddings are rejected before any adapter - #77

Open
ebigunso wants to merge 3 commits into
impl/v0-1-6-w1-task2-port-contractfrom
impl/v0-1-6-w2-task3-record-contract
Open

✨ The vector record becomes a five-field read contract, and zero-norm embeddings are rejected before any adapter#77
ebigunso wants to merge 3 commits into
impl/v0-1-6-w1-task2-port-contractfrom
impl/v0-1-6-w2-task3-record-contract

Conversation

@ebigunso

@ebigunso ebigunso commented Sep 3, 2026

Copy link
Copy Markdown
Owner

Summary

v0.1.6 Wave 2, Task_3 (ADR-I-0025 record read contract, ADR-I-0024 write-side zero-norm rule, ADR-I-0026 published surface policy), stacked on Task_2.

  • The vector record and its typed manifest carry exactly five fields: object id, object type, surface, schema version, and the embedded text as provenance. Relationship, lifecycle, time, ranking, and object-specific hints, the readable text column, the graph URI, and the raw source reference are gone, with their per-field index creation.
  • One Display/FromStr per enum in the domain replaces the adapter's private token mappers and the pipeline's copy.
  • The vector indexing service rejects a zero-norm record embedding as a typed per-record indexing failure before any adapter sees it; one fixture (behind the non-default test-fixtures feature, enabled for integration tests by a self dev-dependency) serves the unit test now and the two-adapter parity suite in Task_4.
  • The maximum number of embedding surfaces per object kind is published as public policy beside the surface policy, with a test that it matches the builders.
  • Database documentation (payload design note, schema cheat sheet, README, graph schema design) describes the five-field record.

Validation

  • Worker (cm-worker2): fmt; Clippy -D warnings on Rust 1.95 and 1.97; service-up cargo test (415 passed, 4 ignored); ignored Qdrant tests 4/4; zero-hit census for dropped fields, content_text readers, and enum-token copies; collection census unchanged.
  • Reviewer (cm-reviewer, Tier D): approved at 36ef8c8 after two revisions on fixture reachability; an independent downstream crate compiled against the feature to prove the fixture is reachable.
  • Cross-repository obligation, not a finding: the evaluation repository's exhaustive error-vocabulary conversion is its own prerequisite for re-pinning (ADR-I-0023 impact).

🤖 Generated with Claude Code

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The reviewed changes have no unresolved approval-blocking issues.

Pull request overview

Simplifies vector records to a five-field read contract and centralizes vector-related domain policy.

Changes:

  • Removes obsolete Qdrant payload fields and indexes.
  • Centralizes persisted enum tokens and surface limits.
  • Rejects zero-norm embeddings before adapter dispatch.
File summaries
File Description
src/usecases/vector_indexing.rs Adds pre-adapter zero-norm rejection and tests.
src/usecases/retrieve.rs Uses the shared domain surface enum.
src/policy/embedding_surface.rs Simplifies builders and publishes surface limits.
src/models/vector/record.rs Defines the five-field vector record.
src/models/vector/candidate_record.rs Adopts the domain surface enum.
src/models/vector.rs Updates exports and adds the shared fixture.
src/lib.rs Exposes surface policy and test fixtures.
src/errors.rs Adds the typed zero-norm error cause.
src/domain.rs Centralizes enums and persisted token conversions.
src/api/types/retrieval.rs Reuses the domain surface type.
src/api/types.rs Removes the duplicate surface export.
src/adapters/qdrant/store.rs Uses canonical tokens and reduced payloads.
src/adapters/qdrant/payload.rs Implements the five-field manifest.
docs/design/database/vector_payload_design.md Documents the new vector contract.
docs/design/database/schema_cheat_sheet.md Updates the compact storage schema.
docs/design/database/README.md Refreshes the vector-design summary.
docs/design/database/graph_schema_design.md Updates cross-store responsibilities.
Cargo.toml Adds the fixture feature and self dev-dependency.
Cargo.lock Records the development dependency change.
Review details
  • Files reviewed: 18/19 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@ebigunso
ebigunso force-pushed the impl/v0-1-6-w2-task3-record-contract branch from 36ef8c8 to d8d0d92 Compare September 3, 2026 19:08
@ebigunso
ebigunso requested a balanced review from Copilot September 3, 2026 19:15

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The broad public and persistence-contract changes warrant final human review.

Review details

Suppressed comments (1)

docs/design/database/schema_cheat_sheet.md:56

  • This says all provenance is hydrated from Oxigraph, but the preceding field list defines embedding_text as embedding provenance retained in Qdrant. Qualify this as source/graph provenance so the five-field contract is not self-contradictory.
These are the only Qdrant payload fields. Readable content, graph URI, object-specific state, relationships, lifecycle/currentness, ranking, timestamps, provenance, and raw references are hydrated from Oxigraph by `object_id`. Existing obsolete extra fields may remain on old points but readers ignore them.
  • Files reviewed: 18/19 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@ebigunso
ebigunso force-pushed the impl/v0-1-6-w2-task3-record-contract branch from d8d0d92 to 7751d90 Compare September 3, 2026 20:49
Copilot AI review requested due to automatic review settings September 3, 2026 20:49

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The reviewed changes have no unresolved approval-blocking issues.

Review details
  • Files reviewed: 18/19 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants