Skip to content

Optional source locator on document observations (page → context, resolvable against the extraction page map) #1366

Description

@phernandez

Follow-up from #1178 (raised by @calebjpicker) and related to #135.

DocumentAgentObservationV1 (in the document ingestion contracts, #1178) is intentionally minimal: category, content, tags, context — no locator. So an extracted quote is traceable to a document and to an exact stored version of it, but not to a place inside it. For legal/academic work that's the difference between a source and a cite ("Exhibit 12" vs "Exhibit 12 at 4:17–22"). The extractor already knows pages (source.page_count, extraction.pages_needing_ocr), and that information is dropped before it reaches an observation.

Shape (deferred from #1178 deliberately, additive to v1)

  • An optional structured locator on the agent observation model — at least a page number, optionally line/offset — that renders deterministically into the existing context parenthetical so the line still round-trips through the real Markdown parser and satisfies require_exact_parser_semantics. (Option 2 from the feat(core): add portable document ingestion contracts #1178 discussion.)
  • A page map on the document/run provenance side (offset→page ranges captured once at extraction) so any later feature can resolve a span to a page without re-parsing the PDF.
  • Optionally carry the source checksum alongside the locator so a citation is verifiable (confirm the quote still matches the extracted bytes; detect drift on re-extraction) — cheap here because the ingestion contract already checksums heavily.

Because DocumentAgentObservationV1 is versioned and extra-forbidden, adding an optional field later is additive, not a breaking migration — which is why #1178 does not need to block on this.

Relationship to #135

#135 asks for section/observation anchor wikilinks at the note level generally. This issue is the document-ingestion instance of it: a PDF page anchor is one concrete kind of observation locator. Whatever addressing scheme #135 settles on, the document locator should use it rather than inventing a parallel one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    cloudBasic Memory CloudenhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions