Skip to content

Future work: prefilter recalls Hindsight memory and/or chat history for 'done-before' check #101

Description

@right-bot

Future work: prefilter recalls Hindsight memory and/or chat history for "done-before" check

Context

Today the skill-learning prefilter (crates/bot/src/learning_prefilter.rs::build_prompt) decides skip / patch_existing / create_new from a snapshot of the just-finished turn:

  • user_msg_text (≤2000 chars)
  • assistant_reply_text (≤4000 chars)
  • used_skill_receipts
  • existing rightx-* skill index (name + ≤120-char description)
  • per-turn cost / num_turns / elapsed vs agent's P50/P90/P99 baseline

That's enough to spot trivial chats and existing-skill reuse, but it has a blind spot: the agent may have solved the same problem days ago without producing a skill receipt, and the prefilter has no signal for that. Result: a create_new decision for a procedure that's already in Hindsight memory or recent chat history, and a redundant probe-writer run downstream.

Proposal

Add a lightweight "have we done this before?" lookup to the prefilter input:

  1. Hindsight recall. Issue a short memory_recall query keyed on the topic of the turn (extracted from the user msg or assistant reply) and pass the top N (≤3) hits into the prompt as PRIOR PROCEDURE EVIDENCE.
  2. Chat history search. Optionally run thread_search / chat_search against the current chat for similar prior turns. Pass back compact snippets (caller, ts, ≤200 chars) so Haiku can see "we already walked the user through this on date X".
  3. Decision impact. Reword the framing so that prior evidence biases toward patch_existing (if a matching skill exists) or skip (if the procedure was handled inline and is too narrow to deserve a skill yet). Keep create_new for genuinely novel work.

Considerations

  • Latency / cost. Recall + search adds round trips and tokens. Cap to single-shot queries with hard byte budgets so the prefilter stays inside its 30s timeout and the Haiku prompt stays small.
  • Daily learning budget. Whatever recall/search costs should be counted against LEARNING_SOURCES so today_spend_usd still gates correctly.
  • False suppression. A naive recall match can suppress legitimately new work that just happens to share keywords. Make the prompt explicit: prior evidence is a hint, not a verdict — Haiku still decides.
  • Schema. No new decision values needed. Optionally extend reason so we can audit which prior-evidence link triggered a patch_existing or skip.

Out of scope

  • Changing the curator / probe-writer side. This is strictly a prefilter input enrichment.
  • Cross-agent recall. Stay scoped to the current agent's own memory and chat for now.

Files

  • crates/bot/src/learning_prefilter.rs — build_prompt, schema extension if needed
  • crates/bot/src/learning_pipeline.rs — wiring of the new input
  • crates/right-agent/src/usage.rs — accounting for the recall/search cost

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions