Skip to content

[P0][LLM] Pass mapper-selected evidence to runtime drivers and record actual use #686

Description

@wesleysimplicio

Objective

Make the actual LLM/runtime invocation consume the Mapper-selected task context instead of calling driver.execute(goal) with only the natural-language goal.

Problem

The distributed context pack contains goal, ACs and source references, but the routed runtime path currently derives a goal string and executes it directly. Exact spans, tests, graph evidence, fidelity and omissions are not delivered to the model, so the model must survey the repository again.

Implementation steps

  1. Define a provider-neutral runtime request carrying goal, ACs, exact Mapper spans, source refs, tests, graph evidence, constraints, omissions and remaining budget.
  2. Add a deterministic renderer for CLI-style drivers with explicit section boundaries and prompt-injection provenance labels.
  3. Keep raw repository instructions/source evidence separated from trusted system/operator constraints.
  4. Apply a final token budget after rendering; required evidence may not be silently truncated.
  5. Pass the structured request to drivers that support it and the rendered request to legacy string-only drivers.
  6. Preserve cwd/tool access while restricting mutation targets to the authorized per-task plan.
  7. Add redaction and secret scanning before dispatch.
  8. Capture a context-consumption receipt: envelope hash, rendered tokens, selected paths/spans, omitted fields, provider/model, cache status and observable token usage.
  9. When providers do not report usage, record null plus reason; never estimate it as measured.
  10. Bind runtime-execution receipt to mapper envelope hash and plan/mutation authority.
  11. Reject stale/mismatched envelopes before calling a provider.
  12. Add compatibility tests for each real driver and malformed/hostile context.

Acceptance criteria

  • The real driver receives goal + ACs + at least one exact source span + relevant verification route in the system test.
  • A legacy string driver receives the deterministic rendered equivalent.
  • Source text cannot override trusted constraints; provenance boundaries are explicit and tested.
  • Secret patterns and unauthorized paths are removed or block dispatch according to policy.
  • Required evidence overflow returns a typed broader-context/budget error, never silent truncation.
  • Runtime receipt binds provider/model/request hash/mapper envelope hash/plan hash.
  • Observable input/output/cache/reasoning tokens are recorded when the provider reports them.
  • Unobservable usage is null with a machine-readable reason.
  • A/B system fixture proves the Mapper path avoids redundant repository survey calls without reducing task correctness.
  • Unit, integration, system, regression and security tests pass.
  • Touched code meets the repository coverage gate.
  • No context or secret is emitted in ordinary logs.

Non-goals

  • Training/fine-tuning a model.
  • Claiming token savings without observed evidence.
  • Allowing the model to expand scope beyond mutation authority.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    Status
    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions