[PIR #1011] Deliver framework files via resolver-aware channels (fresh-install class fix)#1015
Open
amrmelsayed wants to merge 36 commits into
Open
[PIR #1011] Deliver framework files via resolver-aware channels (fresh-install class fix)#1015amrmelsayed wants to merge 36 commits into
amrmelsayed wants to merge 36 commits into
Conversation
…ctor audit) - Layer 2: drop protocol.md pointer from all 9 builder-prompts; fix roles/builder.md cat example; strip A.3 workflow-reference line from spir/protocol.md (both trees) - Patch 2 (A.2): drop redundant plan-template pointers (spir/aspir); embed notes.md / findings.md into experiment/spike protocol.md under drift-guarded sentinels (both trees) - Layer 3: 'Framework files: never shell-fetch by literal path' convention in AGENTS.md + CLAUDE.md; lib/framework-ref-audit.ts + doctor() section (warn, skeleton-only) - Tests: new framework-ref-audit + skeleton-sweep guards; update Spec 746 baselines and bugfix-619 assertion for the swept builder-prompts
Per dev-approval feedback, replace the static embed/append (stale duplicate) with
fresh-at-delivery placeholders:
- spawn-roles.ts: resolveIncludes() ({{> path}}, recursive) + resolveProtocolReference();
buildPromptFromTemplate fills {{protocol_reference}} from protocol.md read fresh at spawn
- builder-prompts: restore 'Read and internalize...'; reference + {{protocol_reference}}
placeholder under {{#if}} (bugfix renders cleanly with neither)
- experiment/spike protocol.md: {{> ...templates...}} include instead of committed copy
- tests: placeholder + include resolution; drift test asserts include-present/no-embed;
re-sweep Spec 746 baselines to the new ## Protocol line
Per dev-approval feedback: the blanket 'never shell-fetch a framework file by literal path' overreached (role docs can be tier-2 overridden into the workspace) and was advisory-not-load-bearing (the doctor check is the mechanical guard). Scope it to builder-facing prompts/role-docs, acknowledge the override, drop the absolute framing.
…ent partial copy #1011's Layer 1 now reliably inlines protocol.md into builders, so the stale bugfix doc reaching builders is a correctness risk. Resolved here: - bugfix/protocol.md: rewrite the stale 548-line manual-flow doc as a concise, porch-accurate ~78-line doc (merge gated by the pr gate, not a manual handshake; drop projectlist + manual architect CMAP; fix branch naming). Ship to the skeleton (was absent) so fresh-install bugfix builders get a correct meta-doc. - experiment/protocol.md: remove the redundant '## notes.md Template' partial copy; the fresh {{> ...}} include already delivers it. Fixes #1013.
The two '**Template**: templates/spec.md|plan.md' annotations and the '## Templates' section pointed builders (via the inlined protocol.md) at template paths absent in fresh installs. The spec/plan structure is delivered by the specify/plan phase prompts; reword to point there and describe templates as skeleton-shipped reference, not on-disk files to fetch. No builder behavior change.
experiment/spike default to mode:soft (no porch phase prompts) -> protocol.md is their only guidance channel, which is why templates are injected into it (and why strict protocols whose phase prompts carry structure are not). Documents the strict-vs-soft distinction and retracts the unneeded 'give them phase prompts' follow-up.
…ser scope) codev doctor is an end-user tool; auditing the global package skeleton was pointless (user can't fix it; CI already guarantees it clean). Scan the project's local codev/protocols + codev/roles overrides instead (no-op when absent), inside the in-a-project block. Shipped-skeleton guard stays in the framework-ref-audit unit test (scans codev-skeleton/, runs in CI). Rename lib param skeletonDir->rootDir; update CLAUDE/AGENTS convention line + plan decision #5.
….md completeness All protocols now ship a protocol.md (bugfix got one in #1013), so the {{#if}} guard was always-true / dead code. Make the protocol reference unconditional in all 9 builder-prompts (both spots, both trees), and add a unit test enforcing that every shipped skeleton protocol with a protocol.json also ships a protocol.md — so the invariant is true-by-construction, not just currently-true. Update the spawn-roles 'absent' test + mock template; re-sweep the Spec 746 baselines.
…d protocols
Closes the residual from removing the {{#if}} guard: a project's own custom/override
protocol with a protocol.json but no protocol.md would render an empty Protocol
Reference section. validateProtocol now warns (non-fatally) in that case, so json-only
protocols stay valid but the omission is flagged at spawn. +2 tests.
…es JSON) via porch-resolved include
Bug: porch's spir/aspir plan gate requires machine-readable phases JSON
(has_phases_json + min_two_phases), which lives only in templates/plan.md. An
earlier change dropped the plan-template pointer treating the inline '### Plan
Structure' as self-contained — but it's JSON-less, so builders would fail the gate.
Fix (Option B):
- Extract include resolution to a shared resolveCodevIncludes in lib/skeleton.ts.
- porch loadPromptFile now resolves {{> }} includes, so phase prompts can pull
framework files fresh (mirrors the spawn-side protocol.md inlining).
- spir/aspir plan.md: replace the divergent inline '### Plan Structure' with
{{> protocols/spir/templates/plan.md}} — builder follows the real template
(with the phases JSON), single-source, fresh.
- spawn-roles uses the shared resolveCodevIncludes.
- Tests: mock provides resolveCodevIncludes; assert plan prompt resolves to carry
the phases JSON, and the template carries it.
The include's template self-documents the required phases JSON (its own 'REQUIRED: porch uses this JSON' comment), so the plan prompt just says 'following the template below:' instead of restating the gate's grep mechanics.
protocol.md is mode-agnostic (soft + strict), so it shouldn't reference the strict-mode 'porch next' delivery. Reword the two annotations I'd added: - spec: '**Structure**: developed through the specify phase' (process-driven) - plan: '**Structure**: follows the plan template provided by the plan phase' No path, no porch internals.
…op cat/cp + padding) Convention is contributor guidance (what to author), not runtime advice — a builder only fetches what a prompt tells it to. State the principle (don't author an instruction that makes the builder read a framework file by literal path; deliver instead) without enumerating shell verbs; the cat/cp specificity lives in the doctor check. Also drop the override-nuance and doctor-pointer sentences; condense the premise.
…NGES iter-1) Remove the spawn-time buildFallbackPrompt: it pointed builders at codev/protocols/... by literal path, re-introducing the fresh-install bug class for protocols without a builder-prompt.md. validateProtocol now hard-errors when none resolves (defensive backstop in buildPromptFromTemplate). Every shipped protocol ships one, so this only affects malformed custom protocols. +2 regression tests.
…T_CHANGES iter-2) The framework-ref audit printed a green success line even for projects with no codev/protocols or codev/roles overrides, contradicting the documented no-op. Gate the audit on a new tested hasFrameworkOverrides predicate so projects with nothing to scan produce no output. +3 unit tests for the predicate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PIR Review: Deliver framework files via resolver-aware channels (fresh-install class fix)
Fixes #1011.
Fixes #1013.
Summary
Post-Spec-618, framework files (protocol docs, role docs, templates, framework resource docs) live only in the package skeleton (resolver tier 4) and are not on disk in fresh installs, yet several builder-facing consumers referenced them by literal
codev/...path (cat/cp/"read this file"), which bypasses the four-tier resolver and fails in a fresh install. This PR is the class fix: it delivers framework files through resolver-aware channels (a{{protocol_reference}}context var that inlinesprotocol.mdfresh at spawn, and a shared{{> path}}include directive resolved on both the spawn and porch phase-prompt channels), sweeps the literal-path references, and adds acodev doctoraudit plus a principle-level convention note as enforcement. It also folds in #1013 (de-stalebugfix/protocol.mdand ship it to the skeleton; drop theexperiment/protocol.mdpartial template copy).Files Changed
48 files, +1113 / -774 (the large deletions are the stale 548-line
bugfix/protocol.mdand the divergent inline plan-structure block). Core code:packages/codev/src/lib/skeleton.ts(+24) — sharedresolveCodevIncludes()packages/codev/src/agent-farm/commands/spawn-roles.ts(+45/-) —{{protocol_reference}}fill via fresh resolver readpackages/codev/src/commands/porch/prompts.ts(+8) — phase-prompt include resolutionpackages/codev/src/lib/framework-ref-audit.ts(+85, new) — shell-fetch auditpackages/codev/src/commands/doctor.ts(+23) — wire audit against workspacecodev/overridesTests:
framework-ref-audit.test.ts(+164, new),spawn-roles.test.ts(+134),bugfix-619-aspir-prompt.test.ts(+10/-), 3 Spec-746 baselines re-swept.Markdown (skeleton + mirrored
codev/): 9 builder-prompts, spir/aspirprompts/plan.md, experiment/spikeprotocol.md, spirprotocol.md,roles/builder.md, new skeletonbugfix/protocol.md(rewritten from the stalecodev/copy),CLAUDE.md/AGENTS.mdconvention note. Plus the plan + thread artifacts.Commits
Grouped by what landed (full per-commit history is on the PR):
protocol.mdat spawn via{{protocol_reference}}; shared{{> path}}include resolution on both the spawn and porch phase-prompt channels (incl. the spir/aspir plan template with its phases JSON).codev/protocols/...references across the 9 builder-prompts,roles/builder.md, andspir/protocol.md.codev doctorframework-ref audit (scoped to workspace overrides, warn-not-error) + a principle-level convention note in CLAUDE.md / AGENTS.md.bugfix/protocol.md(shipped to the skeleton); dropped theexperimentpartial template copy.builder-prompt.md(Codex iter-1);doctortrue no-op for projects with no overrides (Codex iter-2).Test Results
pnpm build: ✓ pass (core then codev, including dashboard + copy-skeleton)pnpm test: ✓ 3281 passed / 13 pre-existing skips on a clean run. New tests cover the{{protocol_reference}}fill,{{> }}include resolution on both spawn and porch channels, the plan template's phases JSON reaching the builder, the doctor audit (positive/negative/no-op), the protocol.md completeness check, thevalidateProtocoljson-without-md warning, the skeleton sweep, the fail-fast behaviour when a protocol ships nobuilder-prompt.md(Codex iter-1 disposition), and thehasFrameworkOverridestrue-no-op predicate (Codex iter-2 disposition).dev-approvalgate and approved. End-to-end against the real skeleton:protocol.mdinlines fresh under "Protocol Reference (full text)", "Read and internalize" restored,{{> }}expands, zero handlebar residue.Architecture Updates
Added a "Builder Prompt: Protocol & Template Delivery (#1011)" subsection to
codev/resources/arch.md(under Agent Farm Internals, next to the role-prompt-injection description). It documents the two new delivery channels ({{protocol_reference}}and the{{> path}}include resolved byresolveCodevIncludeson both the spawn and porch phase-prompt paths) and thecodev doctoraudit, because this PR introduced a genuinely new mechanism for how framework files reach builders.Lessons Learned Updates
Added three bullets to
codev/resources/lessons-learned.md(Protocol Orchestration): deliver framework files via resolver-aware channels rather than fetching them by literal path; soft-mode protocols have onlyprotocol.mdas a guidance channel so a delivery fix must cover both channels; and don't drop a plan-template pointer as a dead reference because the spir/aspir plan template carries the phases JSON porch's plan gate requires.Things to Look At During PR Review
builder-prompt.mdfallback inspawn-roles.tsstill pointed the builder atcodev/protocols/...by literal path, re-introducing the bug class for custom protocols. Real finding. Disposition: rather than harden the fallback (which would duplicate template wording and drift), the fallback was removed in favour of fail-fast.validateProtocolnow hard-errors when a protocol resolves nobuilder-prompt.md, with a defensive backstop inbuildPromptFromTemplate. Every shipped protocol ships abuilder-prompt.md, so this only affects a malformed custom protocol, which now gets a clear error instead of a silently degraded prompt. Pinned by two tests (buildPromptFromTemplatethrows;validateProtocolfails fast on a json-only protocol). PIR is single-pass, so this was not independently re-reviewed: flagged for your attention at theprgate. Claude APPROVE'd; Gemini was skipped non-blockingly (agy not authenticated).codev doctorprinted a✓line even for projects with nocodev/overrides, contradicting the documented "no-op". Addressed (true no-op): the audit is now gated on a testedhasFrameworkOverridespredicate, so a project with no overrides produces no output. Gemini skipped non-blockingly (agy authenticated but timed out producing the review in the builder env, twice — environmental, not auth). Verdicts:codev/projects/1011-*/1011-review-iter2-*.txt.{{> path}}include directive is new surface. It's a Handlebars-style partial resolved by a sharedresolveCodevIncludes()(recursive, depth-guarded at 5, unresolved → empty), run on two channels (spawn-time insideprotocol.md, and porch'sloadPromptFilefor phase prompts). Worth confirming the mechanism choice and the dual-channel wiring.loadPromptFilechange (commands/porch/prompts.ts) is what makes the spir/aspir plan template resolve fresh at the plan phase. The plan gate needs the machine-readable phases JSON (has_phases_json), so this is load-bearing, not cosmetic.cat/cp/…) againstcodev/(protocols|roles)/..., runs against the project's localcodev/overrides (end-user-controlled), warn-not-error; the shipped skeleton is guarded by a CI unit test instead.codev-skeleton/and the self-hostedcodev/tree. Filed scaffold: CI test for codev/ ↔ codev-skeleton/ parity (catch drift when only one tree is edited) #1016 to add a CI parity guard.How to Test Locally
For reviewers pulling the branch:
afx dev pir-1011(or the VSCode Run Dev Server action)porch next <id>for a spir/aspir project at the plan phase contains the plan template's"phases"JSON and no literal{{>.codev initscratch project, thenafx spawn --task ... --protocol spir: the spawned builder's opening prompt contains the inlinedprotocol.mdunder "Protocol Reference (full text)" with zero{{residue (tier-4 skeleton fallback path).codev doctorin a project with acodev/override containing acat codev/protocols/...line flags it (warn), and is a no-op with no overrides.Flaky Tests
Two pre-existing tests flaked once under full-suite parallel load (5s timeouts), then passed on a clean re-run of the same code:
packages/codev/src/__tests__/team-cli.test.ts:140("auto-detects author when not provided")packages/codev/src/__tests__/team-github.test.ts:241("returns gracefully when gh CLI succeeds or fails")Both contend on the
ghCLI / git-author subprocess under parallelism and are unrelated to this diff (which touches noteam-*code). Not fixed or quarantined per protocol scope rules.