feat: finish Broad-Side — Pi surface, scout-first pipeline, per-lens models, and the missing paper trail (#138, #139, #141) - #154
Open
TheAmericanMaker wants to merge 4 commits into
Conversation
…ture
Broad-Side shipped across seven PRs with its user-facing paper trail
lagging the code. Three gaps, one of them a defect:
- .codecarto/broadside/SKILL.md was written but unreachable. codecarto_skill
resolves .codecarto/skills/<name>/SKILL.md, so {name: "broadside"} returned
"Unknown skill" and codecarto_list_skills never mentioned it. Both surfaces
now serve it under the name `broadside` via the new readBroadsideSkill,
exempt from the post-pipeline completion gate — a scout run is read before
the pipeline and during it, not after — and readable on a repository with
scout state and no workspace (the packaged copy answers when there is no
workspace copy). list_skills reports it apart from the post-pipeline set,
and the unknown-skill error names the exemption.
- The packaged agent skill never mentioned Broad-Side, so an agent with the
tool had no doctrine for it. New references/broadside.md covers when to
scout, the cost guardrails, how to read a run, and the leads-never-evidence
rule; it is served as the `broadside` topic of codecarto_guide and linked
from the skill's drive loop and reference list.
- config.yaml lagged the tool: incremental, retry_truncated,
include_synthesis, include_triage and wait_seconds were call-parameters
only, so a repo could not fix its own scouting policy. All five are now
config keys with repo defaults. An explicit tool parameter still wins, and
a malformed value falls back to the shipped default rather than failing a
run mid-flight.
Also: a README section, a MANUAL step, and MCP-quickstart coverage in place
of a single table row; broadside/ in the README repo-structure block and
core/broadside.ts in CLAUDE.md's core-modules list; and npm run
smoke:broadside for the previously orphaned opt-in live smoke script.
Tests: config defaults and their malformed-value fallbacks, a check that the
documented config keys and the parsed config keys are the same set, skill
reachability on both the MCP and Pi surfaces, and the guide reference's
content rules. 413 pass.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ace (#138) Broad-Side shipped MCP-first by agreement, leaving the recommended user surface without it. The Pi command exposes the same four actions with tab-completion for actions, lens names, and flags, and a live per-lens progress widget while batches poll. Two divergences from MCP, both deliberate: - The spend decision is interactive. MCP cannot ask a human, so it refuses a run over max_cost until the caller passes force. Pi shows the per-lens breakdown, the rates, the limit and whether the run exceeds it, then asks — and an approval IS the force flag. runBroadsideSubmit grows an optional confirm hook that fires after slicing and before any state write or submission; declining throws BroadsideCancelledError and nothing is submitted. Surfaces with no human omit the hook and keep the old path. - The command takes no API key argument. A key typed into a slash command lands in the session transcript, so it is OPENROUTER_API_KEY or config.yaml only, and the error message says why. Like the MCP tool it runs on a repository with no workspace; there the result renders into its own widget rather than the phase widget, which has no state to draw. Argument parsing lives in broadside-flags.ts following the parseNextFlags idiom: never throws, collects unknown tokens, and refuses flags that mean nothing for the chosen action rather than ignoring them — silently dropping --incremental on a collect would read as "collected incrementally". Tests: the confirm hook's decline-and-approve paths at the core level, the flag grammar including a check that every completion token the command offers is one the parser accepts, and the command end-to-end through a fake Pi harness with stubbed fetch. Two new invariants: the extension must register a command for every framework operation (broadside included), and the README table must name every registered command. 430 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
) Broad-Side produced reports nothing in a pipeline pointed at. The scout-first variant is full-with-deep-audit with a broadside-scout phase in front. Two constraints shaped the design, and both are worth stating: - A required_read must be an upstream phase's primary_output, and run directories are timestamped and gitignored, so no pipeline YAML can ever name broadside/<run>/synthesis.md. The scout phase therefore distills: it reads whatever a prior run wrote and produces one stable, reviewable artifact, findings/broadside-scout/scout-brief.md. That indirection is also where the judgment lives — deciding which leads are worth six phases' attention is not a file copy. - status.yaml phases must match phase_order exactly, so a phase cannot be optional. Scouting therefore ships as its own variant rather than as a toggle on the default pipeline, which would otherwise put a paid API scan in front of every run. Architecture, defect-scan-mechanical, contracts, protocols, defect-scan-semantic and porting read the brief, and each carries a completion criterion requiring every lead routed to it to be confirmed against the source, dismissed with a reason, or carried forward — never reported as a finding on the brief's authority. That criterion is where "leads, never evidence" stops being a docstring and becomes a gate. reimplementation-spec deliberately does not read the brief; the porting bundle stays its compression boundary. The phase never submits a batch and never spends. With no run on disk it writes an explicitly empty brief with coverage disposition NONE and the pipeline proceeds exactly as full-with-deep-audit would — which matters because /codecarto-next --auto runs it unattended. Tests pin who reads the brief, who must account for it, that the spec phase does not, that the skill never tells an executor to fire a run, and that the variant matches the deep-audit pipeline it wraps field by field apart from the scout additions. 441 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The multi-model item was half-built: the catalog, the models action, and the pricing/capability pre-flight all shipped, but a run still had exactly one model for all six lenses. That is the wrong granularity. A stronger model changes what the security and defect lenses find far more than it changes an architecture map, so the useful knob is per lens, not per run. lens_models in config.yaml routes individual lenses to their own batch model. Each distinct model is resolved and pre-flighted independently: priced from the live catalog, refused without structured-output support, and clamped to its own completion ceiling. The estimate carries a model and its rates per lens plus a mixedModels flag; the Pi confirmation names the overridden model on the affected rows, so a mixed-model run cannot be approved without seeing which lens costs what. Submission now fires from the priced rows rather than recomputing them — the request that goes out is the one the user approved. run-meta.json records lens_models, and collect's truncation retry re-submits on the lens's own model and ceiling rather than the run default, which would otherwise change the model mid-run. An override naming an unknown lens id is dropped, not carried: it can only be a typo, and a key that looks applied but is not is worse than one that never appears. What this deliberately does NOT do is change the default. The roadmap's remaining item was "a stronger default for semantic lenses," and that is a comparative-evaluation question on real repositories with real spend — picking one here would spend every user's money on our guess. The config says so and points at the models action. Per-model prompt tweaks stay open for the same reason: they need the same evidence. Tests: override parsing including the dropped-typo case, and an end-to-end run asserting the override reaches the estimate with its own rates, both levels of the batch payload, and the recorded per-lens entry, while the other lens stays on the default. The config-key drift test now separates top-level keys from nested examples instead of conflating them. 443 pass. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Finishes Broad-Side: the surfaces, docs, and pipeline integration that the seven feature PRs left behind, plus the half of multi-model that can be decided without spending money.
Stacked on #153. Base is
feat/142-broadside-incremental-rescoutso the diff shows only this work — please retarget tomainonce #153 merges.What this closes
9d9c18fa18d394d3da986scout-firstpipeline (#139)fd179e3The defect
.codecarto/broadside/SKILL.mdwas written but unreachable.codecarto_skillresolves.codecarto/skills/<name>/SKILL.md, so{name: "broadside"}returned "Unknown skill" andcodecarto_list_skillsnever mentioned it. Both surfaces now serve it under the namebroadside, exempt from the post-pipeline completion gate — a scout run is read before the pipeline and during it — and readable on a repository with scout state and no workspace.Pi surface (#138)
/codecarto-broadside [submit|collect|status|models] [lenses…]with tab-completion for actions, lenses, and flags, and a live per-lens progress widget. Two deliberate divergences from MCP:max_costuntil the caller passesforce. Pi shows the per-lens breakdown, the rates, and whether the run exceeds the limit — and an approval is the force flag. This needed aconfirmhook onrunBroadsideSubmit, firing after slicing and before any state write; declining throwsBroadsideCancelledErrorand nothing is submitted. Surfaces with no human omit the hook and keep the old path unchanged.OPENROUTER_API_KEYorconfig.yamlonly, and the error says why.Pipeline phase (#139)
Two constraints shaped this, and neither is what the roadmap item assumed:
required_readsentry must be an upstream phase'sprimary_output, and run directories are timestamped and gitignored — so no pipeline YAML can ever namebroadside/<run>/synthesis.md. Thebroadside-scoutphase therefore distills: it reads what a prior run wrote and produces one stable artifact,findings/broadside-scout/scout-brief.md. That indirection is also where the judgment lives — deciding which leads are worth six phases' attention is not a file copy.status.yamlphases must matchphase_orderexactly, so a phase cannot be optional. Scouting ships as its own variant (pipeline-scout-first.yaml, aliasscout-first) rather than as a toggle on the default, which would put a paid API scan in front of every run.Architecture, both defect scans, contracts, protocols, and porting read the brief, and each carries a completion criterion requiring every lead routed to it to be confirmed against the source, dismissed with a reason, or carried forward — never reported as a finding on the brief's authority. That criterion is where "leads, never evidence" stops being a docstring and becomes a gate.
reimplementation-specdeliberately does not read it; the porting bundle stays its compression boundary.The phase never submits a batch and never spends. With no run on disk it writes an explicitly empty brief and the pipeline proceeds exactly as
full-with-deep-auditwould — which matters because/codecarto-next --autoruns it unattended.Per-lens models (#141, mechanism only)
lens_modelsroutes individual lenses to their own batch model: a stronger model changes security and defect findings far more than it changes an architecture map. Each distinct model is resolved and pre-flighted independently, the estimate carries per-lens rates, submission fires from the priced rows rather than recomputing them, and collect's truncation retry re-submits on the lens's own model and ceiling instead of the run default.Not shipped, on purpose: a stronger default for the semantic lenses. That is a comparative-evaluation question on real repositories with real spend, and choosing one here would spend every user's money on our guess. The config, ROADMAP, and CHANGELOG all say so and point at the
modelsaction. Per-model prompt tweaks stay open for the same reason.Also
config.yamlgained repo defaults for every per-call run knob (incremental,retry_truncated,include_synthesis,include_triage,wait_seconds) — previously call-parameters only, so a repo could not fix its own scouting policy.references/broadside.mdin the packaged agent skill, served as thebroadsidetopic ofcodecarto_guide.npm run smoke:broadsidewires the previously orphaned opt-in live smoke script.Verification
443 tests pass (was 405),
npm run buildclean. New invariants worth noting: the documented and parsed config-key sets must match; the extension must register a command for every framework operation; the README table must name every registered command; andpipeline-scout-first.yamlis pinned field-by-field to the deep-audit pipeline it wraps, so the two cannot diverge silently.🤖 Generated with Claude Code