Transplant reqdrive's prompt discipline into the engine (W2) - #70
Merged
Conversation
… (W2)
The engine ran on two-to-three sentence prompts. steps.rs was 116 lines, and the
convergence plan called W2 "the single highest-leverage de-risk of the 0%-proven
live path" and W2 + W4 together "the whole thesis". W4 landed in July; W2 never
did, so the engine advanced on real evidence while being driven by the stubs the
plan set out to replace.
What ported is the discipline, not the text:
oracle must fail against the current tree - a test that already passes
defines nothing; assert observable behaviour, not names it is about to
invent; ordinary case plus a boundary plus an implied failure; record
ambiguity as a comment. It is hashed and frozen the moment it finishes,
so a vague oracle is not a weak test but a wrong definition of done
that nothing downstream can correct.
build one thing, smallest change, address findings first, run the real test
command rather than claiming success, never edit a frozen test - and,
unlike reqdrive, do NOT commit: this harness owns version control, and
an agent that commits corrupts the diff review and merge read.
review reqdrive's four named criteria (security, correctness, scope, quality)
in place of "review for correctness and quality", plus the standing
reminder that a green suite is not evidence, since the tests predate
the code.
What did NOT port, and why it matters: reqdrive's prompts are welded to its own
artifacts - prd.json, progress.txt, .reqdrive/runs/, per-story ids, the
iteration-summary block. This harness has none of them and no story concept at
all. So W2 as the plan worded it DEPENDS ON W6, which the plan schedules later.
That dependency was never stated. The transferable half is here; the
story-shaped half waits on a story model.
Five prompt-invariant tests. A prompt is the least reviewable thing in the
engine - no type, no compiler - so the rules that make it safe are asserted
rather than trusted. They pin the discipline, not the wording; reword freely,
but drop a rule and they fail. Each was driven red:
no-commit rule removed -> builder test FAILED
BLOCKERS contract removed -> review test FAILED
oracle told tests may pass -> oracle test FAILED
Also found and documented, not fixed: parse_blockers treats an ABSENT marker as
zero, so a reviewer that crashed or wrote prose scores clean and satisfies the
review gate - stronger than allowing the turn, it actively passes it. Under I1
as amended these are two different failures and must not share an outcome. The
fix is Option<u32> with None routed to NeedsHuman, a state-machine change that
does not belong in a prompt commit; it is asserted as a named known gap so it is
a decision rather than a surprise. The prompt half of the mitigation is in place.
fleetd 87 -> 92 tests. Workspace check clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The engine ran on two-to-three sentence prompts —
steps.rswas 116 lines. The convergence plan called W2 "the single highest-leverage de-risk of the 0%-proven live path" and W2 + W4 together "the whole thesis." W4 landed in July; W2 never did. So the engine advances on real evidence while being driven by the stubs the plan set out to replace.What ported — the discipline, not the text
What did not port, and why it matters
reqdrive's prompts are welded to its own artifacts —
prd.json,progress.txt,.reqdrive/runs/, per-story ids, theiteration-summaryblock. This harness has none of them and no story concept at all.So W2 as the plan worded it depends on W6, which the plan schedules later. That dependency was never stated. The transferable half is here; the story-shaped half waits on a story model.
Prompt-invariant tests
A prompt is the least reviewable thing in the engine — no type, no compiler — so the rules that make it safe are asserted rather than trusted. They pin the discipline, not the wording. Each driven red:
Found and documented, deliberately not fixed
parse_blockerstreats an absent marker as zero. A reviewer that crashed, ran out of budget mid-sentence, or wrote prose scores clean and satisfies the review gate — stronger than allowing the turn, it actively passes it.Under I1 as amended these are two different failures — a check that ran and found nothing versus one that could not report — and they must not share an outcome. The fix is
Option<u32>withNone→NeedsHuman: a state-machine change that doesn't belong in a prompt commit. It's asserted as a named known gap so it's a decision rather than a surprise, and the prompt half of the mitigation is in place.fleetd 87 → 92 tests. Workspace check clean. Touches different files from #68, so they merge in either order.
🤖 Generated with Claude Code