Skip to content

(MOT-4162) harness: minimal identity prompt + on-demand agent playbook skills - #558

Open
rohitg00 wants to merge 6 commits into
mainfrom
harness-minimal-prompt
Open

(MOT-4162) harness: minimal identity prompt + on-demand agent playbook skills#558
rohitg00 wants to merge 6 commits into
mainfrom
harness-minimal-prompt

Conversation

@rohitg00

@rohitg00 rohitg00 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

What

  • prompts/default.txt shrinks from ~9,450 to 1,676 tokens (tokenizer-measured via context::count-tokens). The universal call loop, discovery surface, payload and error rules, security, and conventions stay.
  • The orchestration, finishing, and building playbooks move verbatim into skills/orchestration.md (3,686 tok), skills/finishing.md (2,362 tok), and skills/building.md (1,172 tok). iii-directory already serves worker-shipped skills, so agents pull them as harness/orchestration, harness/finishing, harness/building. The prompt instructs: pull the matching playbook BEFORE the first action of that kind, once per session.
  • prompt/tests.rs keeps every content invariant. Tests for moved sections now pin the skill files via include_str!, so the doctrine stays test-locked exactly as before; two new tests pin the skill pointers and playbook contents.
  • The orchestration playbook carries the reaction-session rule that live testing exposed, in the form the runtime now has after (MOT-4184) feat(harness): add prompt evaluation support #595: an ordinary event reaction MUST omit metadata.session_id, and every event then creates a fresh distinct child under the resolved console root.

Why

  • 63% of the identity prompt is the orchestration playbook. Sessions that never spawn pay it on every generation.
  • Seven provider workers ship near-copies of the same prompt (28 to 32 KB each, already drifted apart). Moving the bulk into versioned skills gives one canonical copy; a follow-up PR points the provider identities at the same minimal core.
  • The fp worker already proved the pattern in the other direction: its pipe guidance moved out of the prompt variants into a presence-gated pre-generate hook. This PR does the same for the playbooks, with on-demand pull instead of per-turn injection.

Test evidence

Live A/B on a rig (engine 0.21.6, harness 1.4.2, claude-sonnet-5, 13 sessions): identical task set, baseline identity prompt vs this minimal core via the router operator override.

metric baseline minimal core
system tokens per generation 7,850 2,371 (fp hook injection included in both)
simple task (ls, workers query, registry lookup) correct correct, zero skill pulls, fewer turns
fan-out task (3 children, state writes, report) correct correct; agent pulled harness/orchestration before its first engine::register_trigger and harness::spawn, unprompted
conventions (FAILED: first line, @fn pills) partial held

Note: the rig ran harness 1.4.2, where an in-turn spawn still parks and returns the child result; the playbook text intentionally keeps the 1.5.x fire-and-forget doctrine and was not weakened by that observation.

Rollout

  • default.txt is the router-absent fallback, so this PR alone changes behavior only on rigs without a provider identity prompt.
  • Skills ship with the bundle; iii-directory picks them up on install (auto_download covers new adds).
  • Follow-ups: provider identity prompts unify on the same core; iii-directory gains an optional pre-generate hook injecting a compact skills index for automatic discovery.

Draft: direction review wanted before the provider prompts move.

Linear

Closes MOT-4162. Part of MOT-4157.

Coordination

Update: merged main (2026-07-27)

main moved under this branch while it sat open, so it is merged in rather than left conflicting. Three things came out of that:

  • The curl prohibition is gone from the identity prompt. The web worker has injected that rule through its own presence-gated pre-generate hook since feat(web): self-inject web::fetch guidance via pre-generate hook #385, so the line in default.txt was a stale duplicate. It also fired on rigs without the web worker installed, where it names a capability that is not there.
  • (MOT-4184) feat(harness): add prompt evaluation support #595's dispatch doctrine follows the rest of the playbook text. Auditing every resolved resource selector before dispatch, and deriving a per-run namespace suffix after confirming the namespace is absent, now live in skills/orchestration.md alongside the spawn mechanics they belong to.
  • (MOT-4184) feat(harness): add prompt evaluation support #595's corrected reaction semantics replace the older rule. The playbook previously said omitting metadata.session_id delivers back into the registering chat. The runtime no longer behaves that way, so the playbook was describing behaviour that had already changed. once is likewise documented as a top-level registration field that is standing by default on join predecessor edges.

The three tests #595 added assert that doctrine against the identity prompt. They now assert it against the orchestration playbook, which is where the text lives after this branch. cargo test --all-features is green at 295 lib tests, and cargo fmt --check and cargo clippy --all-targets --all-features -- -D warnings both pass.

Summary by CodeRabbit

  • Documentation

    • Streamlined agent guidance with clearer function discovery, contract retrieval, JSON-object payload requirements, and categorized error-handling with retry/fix instructions.
    • Added on-demand agent playbooks for orchestration, finishing fan-out runs, and building/registering workers, including stricter trigger and completion-wiring expectations.
    • Updated trigger and finish/checklist guidance, including wake-up invariants, deadline fallbacks, and SDK/documentation reference requirements.
  • Tests

    • Updated prompt validation to match the new playbook-based workflow and revised documentation references.

@vercel

vercel Bot commented Jul 22, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
workers Ready Ready Preview, Comment Jul 27, 2026 1:21pm
workers-tech-spec Ready Ready Preview, Comment Jul 27, 2026 1:21pm

Request Review

@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The default harness prompt now enforces contract-first function calls and delegates orchestration, finishing, and building guidance to on-demand skill playbooks. Prompt tests load these playbooks directly and validate their key invariants.

Changes

Harness playbook extraction

Layer / File(s) Summary
Prompt contract and skill dispatch
harness/prompts/default.txt, harness/skills/SKILL.md
Function discovery, contract retrieval, JSON-object payloads, categorized retries, security rules, and on-demand skill loading are consolidated in the prompt and skill index.
Orchestration and finishing playbooks
harness/skills/orchestration.md, harness/skills/finishing.md
Playbooks define trigger-based fan-out, reactions, joins, completion signaling, deadlines, and end-of-run handling.
Worker building workflow
harness/skills/building.md
The building playbook documents worker discovery and installation, coder-backed edits, worker registration, and SDK reference requirements.
Playbook and prompt invariants
harness/src/prompt/tests.rs
Tests load playbooks directly and validate prompt allowlists, skill pointers, workflow guidance, SDK references, orchestration, finishing, and capability ordering.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

  • iii-hq/workers#460: Updates related function discovery, contract retrieval, and denied-function prompt guidance.
  • iii-hq/workers#540: Introduces orchestration semantics reflected in these harness playbooks.

Poem

I’m a rabbit with playbooks tucked neat,
Contracts and triggers now march to a beat.
Fan-outs hop onward, completion shines bright,
Builders read SDKs before taking flight.
Tests guard each burrow—what a delightful sight!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: a reduced harness identity prompt plus on-demand agent playbook skills.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch harness-minimal-prompt

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

skill-check — worker

0 verified, 49 skipped (no docs/).

Layer Result
structure
vale
ai
render

Four for four. Nicely done.

The docs moved SDK references from /docs/api-reference/sdk-* to
/docs/reference/sdk-*, and the engine wire protocol page lives at
/docs/reference/engine-protocol; the old paths 404. Verified live:
old paths return 404, new paths return 200 and serve markdown with
the .md suffix. Updated the playbook and the prompt invariants that
pin the URLs.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@harness/prompts/default.txt`:
- Around line 21-35: Update the discovery and contract-retrieval instructions in
Steps 1–2 to apply only to undocumented function ids. Explicitly exempt
documented calls such as directory::skills::get, registry calls, and
harness::react trigger targets, while preserving the existing discovery
requirements for all other functions and the documented batching and re-fetch
exceptions.

In `@harness/skills/finishing.md`:
- Around line 21-26: Update Step 3 in the validator registration guidance to
limit parent_session_id filters to children spawned within the current turn. For
trigger-fired or direct-spawn fan-outs without parent links, require per-child
session_id subscriptions or state-key joins so completions reach the validator
and turn_complete is written.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 99bdfe6e-1087-42d2-8041-d3ab40a3180f

📥 Commits

Reviewing files that changed from the base of the PR and between 53ef04c and ae9db57.

📒 Files selected for processing (6)
  • harness/prompts/default.txt
  • harness/skills/SKILL.md
  • harness/skills/building.md
  • harness/skills/finishing.md
  • harness/skills/orchestration.md
  • harness/src/prompt/tests.rs

Comment thread harness/prompts/default.txt Outdated
Comment thread harness/skills/finishing.md
…caveat

Step 2 now states explicitly that functions documented in the prompt
(the discovery functions and directory::skills::get) skip the find-id
and get-contract steps; everything else keeps the full discipline.

finishing.md Step 3 documents the parent_session_id boundary: the
filter only matches children that carry the parent link (direct spawns
from the session, or reaction spawns whose metadata names it).
Reaction- or wave-dispatched children without the link need the
run-scope state keys or an explicit per-child join, otherwise their
completions never reach the validator and turn_complete is never
written.
Resolves against the prompt-evaluation work that landed in #595, which
edited the same sections this branch moves out of the identity prompt.

- `default.txt` keeps this branch's trimmed identity. The `curl`
  prohibition is dropped with it: the web worker has injected that rule
  through its own pre-generate hook since #385, so the line was a stale
  duplicate that also fired on rigs without the web worker installed.
- #595's dispatch doctrine (audit every resolved resource selector before
  dispatching; derive a per-run namespace suffix and confirm the namespace
  is absent) moves into `harness/skills/orchestration.md`, which now owns
  spawn and reaction mechanics.
- #595 also corrected the reaction session semantics: an ordinary event
  reaction MUST omit `metadata.session_id`, and every event then creates a
  fresh distinct child under the resolved console root. The playbook still
  carried the older rule, that omitting it delivers back into the
  registering chat, so it was describing behaviour the runtime no longer
  has. Same for `once`, which is a top-level registration field and is
  standing by default on join predecessor edges.
- The three tests #595 added assert that doctrine against the identity
  prompt. They now assert it against the orchestration playbook, which is
  where the text lives after this branch.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant