Skip to content

feat: finish Broad-Side — Pi surface, scout-first pipeline, per-lens models, and the missing paper trail (#138, #139, #141) - #154

Open
TheAmericanMaker wants to merge 4 commits into
feat/142-broadside-incremental-rescoutfrom
fix/broadside-skill-reachability-and-docs
Open

feat: finish Broad-Side — Pi surface, scout-first pipeline, per-lens models, and the missing paper trail (#138, #139, #141)#154
TheAmericanMaker wants to merge 4 commits into
feat/142-broadside-incremental-rescoutfrom
fix/broadside-skill-reachability-and-docs

Conversation

@TheAmericanMaker

Copy link
Copy Markdown
Member

Finishes Broad-Side: the surfaces, docs, and pipeline integration that the seven feature PRs left behind, plus the half of multi-model that can be decided without spending money.

Stacked on #153. Base is feat/142-broadside-incremental-rescout so the diff shows only this work — please retarget to main once #153 merges.

What this closes

Commit Item
9d9c18f Reading guide reachable + the feature documented
a18d394 Pi extension surface (#138)
d3da986 scout-first pipeline (#139)
fd179e3 Per-lens model overrides (#141, mechanism half)

The defect

.codecarto/broadside/SKILL.md was written but unreachable. codecarto_skill resolves .codecarto/skills/<name>/SKILL.md, so {name: "broadside"} returned "Unknown skill" and codecarto_list_skills never mentioned it. Both surfaces now serve it under the name broadside, exempt from the post-pipeline completion gate — a scout run is read before the pipeline and during it — and readable on a repository with scout state and no workspace.

Pi surface (#138)

/codecarto-broadside [submit|collect|status|models] [lenses…] with tab-completion for actions, lenses, and flags, and a live per-lens progress widget. Two deliberate divergences from MCP:

  • The spend decision is interactive. MCP cannot ask a human, so it refuses a run over max_cost until the caller passes force. Pi shows the per-lens breakdown, the rates, and whether the run exceeds the limit — and an approval is the force flag. This needed a confirm hook on runBroadsideSubmit, firing after slicing and before any state write; declining throws BroadsideCancelledError and nothing is submitted. Surfaces with no human omit the hook and keep the old path unchanged.
  • No API key argument. A key typed into a slash command lands in the session transcript, so it is OPENROUTER_API_KEY or config.yaml only, and the error says why.

Pipeline phase (#139)

Two constraints shaped this, and neither is what the roadmap item assumed:

  • A required_reads entry must be an upstream phase's primary_output, and run directories are timestamped and gitignored — so no pipeline YAML can ever name broadside/<run>/synthesis.md. The broadside-scout phase therefore distills: it reads what a prior run wrote and produces one stable artifact, findings/broadside-scout/scout-brief.md. That indirection is also where the judgment lives — deciding which leads are worth six phases' attention is not a file copy.
  • status.yaml phases must match phase_order exactly, so a phase cannot be optional. Scouting ships as its own variant (pipeline-scout-first.yaml, alias scout-first) rather than as a toggle on the default, which would put a paid API scan in front of every run.

Architecture, both defect scans, contracts, protocols, and porting read the brief, and each carries a completion criterion requiring every lead routed to it to be confirmed against the source, dismissed with a reason, or carried forward — never reported as a finding on the brief's authority. That criterion is where "leads, never evidence" stops being a docstring and becomes a gate. reimplementation-spec deliberately does not read it; the porting bundle stays its compression boundary.

The phase never submits a batch and never spends. With no run on disk it writes an explicitly empty brief and the pipeline proceeds exactly as full-with-deep-audit would — which matters because /codecarto-next --auto runs it unattended.

Per-lens models (#141, mechanism only)

lens_models routes individual lenses to their own batch model: a stronger model changes security and defect findings far more than it changes an architecture map. Each distinct model is resolved and pre-flighted independently, the estimate carries per-lens rates, submission fires from the priced rows rather than recomputing them, and collect's truncation retry re-submits on the lens's own model and ceiling instead of the run default.

Not shipped, on purpose: a stronger default for the semantic lenses. That is a comparative-evaluation question on real repositories with real spend, and choosing one here would spend every user's money on our guess. The config, ROADMAP, and CHANGELOG all say so and point at the models action. Per-model prompt tweaks stay open for the same reason.

Also

  • config.yaml gained repo defaults for every per-call run knob (incremental, retry_truncated, include_synthesis, include_triage, wait_seconds) — previously call-parameters only, so a repo could not fix its own scouting policy.
  • references/broadside.md in the packaged agent skill, served as the broadside topic of codecarto_guide.
  • README section, MANUAL step, MCP-quickstart coverage, in place of a single table row.
  • npm run smoke:broadside wires the previously orphaned opt-in live smoke script.

Verification

443 tests pass (was 405), npm run build clean. New invariants worth noting: the documented and parsed config-key sets must match; the extension must register a command for every framework operation; the README table must name every registered command; and pipeline-scout-first.yaml is pinned field-by-field to the deep-audit pipeline it wraps, so the two cannot diverge silently.

🤖 Generated with Claude Code

TheAmericanMaker and others added 4 commits August 30, 2026 15:29
…ture

Broad-Side shipped across seven PRs with its user-facing paper trail
lagging the code. Three gaps, one of them a defect:

- .codecarto/broadside/SKILL.md was written but unreachable. codecarto_skill
  resolves .codecarto/skills/<name>/SKILL.md, so {name: "broadside"} returned
  "Unknown skill" and codecarto_list_skills never mentioned it. Both surfaces
  now serve it under the name `broadside` via the new readBroadsideSkill,
  exempt from the post-pipeline completion gate — a scout run is read before
  the pipeline and during it, not after — and readable on a repository with
  scout state and no workspace (the packaged copy answers when there is no
  workspace copy). list_skills reports it apart from the post-pipeline set,
  and the unknown-skill error names the exemption.

- The packaged agent skill never mentioned Broad-Side, so an agent with the
  tool had no doctrine for it. New references/broadside.md covers when to
  scout, the cost guardrails, how to read a run, and the leads-never-evidence
  rule; it is served as the `broadside` topic of codecarto_guide and linked
  from the skill's drive loop and reference list.

- config.yaml lagged the tool: incremental, retry_truncated,
  include_synthesis, include_triage and wait_seconds were call-parameters
  only, so a repo could not fix its own scouting policy. All five are now
  config keys with repo defaults. An explicit tool parameter still wins, and
  a malformed value falls back to the shipped default rather than failing a
  run mid-flight.

Also: a README section, a MANUAL step, and MCP-quickstart coverage in place
of a single table row; broadside/ in the README repo-structure block and
core/broadside.ts in CLAUDE.md's core-modules list; and npm run
smoke:broadside for the previously orphaned opt-in live smoke script.

Tests: config defaults and their malformed-value fallbacks, a check that the
documented config keys and the parsed config keys are the same set, skill
reachability on both the MCP and Pi surfaces, and the guide reference's
content rules. 413 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ace (#138)

Broad-Side shipped MCP-first by agreement, leaving the recommended user
surface without it. The Pi command exposes the same four actions with
tab-completion for actions, lens names, and flags, and a live per-lens
progress widget while batches poll.

Two divergences from MCP, both deliberate:

- The spend decision is interactive. MCP cannot ask a human, so it refuses
  a run over max_cost until the caller passes force. Pi shows the per-lens
  breakdown, the rates, the limit and whether the run exceeds it, then asks
  — and an approval IS the force flag. runBroadsideSubmit grows an optional
  confirm hook that fires after slicing and before any state write or
  submission; declining throws BroadsideCancelledError and nothing is
  submitted. Surfaces with no human omit the hook and keep the old path.

- The command takes no API key argument. A key typed into a slash command
  lands in the session transcript, so it is OPENROUTER_API_KEY or
  config.yaml only, and the error message says why.

Like the MCP tool it runs on a repository with no workspace; there the
result renders into its own widget rather than the phase widget, which has
no state to draw.

Argument parsing lives in broadside-flags.ts following the parseNextFlags
idiom: never throws, collects unknown tokens, and refuses flags that mean
nothing for the chosen action rather than ignoring them — silently dropping
--incremental on a collect would read as "collected incrementally".

Tests: the confirm hook's decline-and-approve paths at the core level, the
flag grammar including a check that every completion token the command
offers is one the parser accepts, and the command end-to-end through a fake
Pi harness with stubbed fetch. Two new invariants: the extension must
register a command for every framework operation (broadside included), and
the README table must name every registered command. 430 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
)

Broad-Side produced reports nothing in a pipeline pointed at. The scout-first
variant is full-with-deep-audit with a broadside-scout phase in front.

Two constraints shaped the design, and both are worth stating:

- A required_read must be an upstream phase's primary_output, and run
  directories are timestamped and gitignored, so no pipeline YAML can ever
  name broadside/<run>/synthesis.md. The scout phase therefore distills:
  it reads whatever a prior run wrote and produces one stable, reviewable
  artifact, findings/broadside-scout/scout-brief.md. That indirection is
  also where the judgment lives — deciding which leads are worth six
  phases' attention is not a file copy.

- status.yaml phases must match phase_order exactly, so a phase cannot be
  optional. Scouting therefore ships as its own variant rather than as a
  toggle on the default pipeline, which would otherwise put a paid API scan
  in front of every run.

Architecture, defect-scan-mechanical, contracts, protocols,
defect-scan-semantic and porting read the brief, and each carries a
completion criterion requiring every lead routed to it to be confirmed
against the source, dismissed with a reason, or carried forward — never
reported as a finding on the brief's authority. That criterion is where
"leads, never evidence" stops being a docstring and becomes a gate.
reimplementation-spec deliberately does not read the brief; the porting
bundle stays its compression boundary.

The phase never submits a batch and never spends. With no run on disk it
writes an explicitly empty brief with coverage disposition NONE and the
pipeline proceeds exactly as full-with-deep-audit would — which matters
because /codecarto-next --auto runs it unattended.

Tests pin who reads the brief, who must account for it, that the spec phase
does not, that the skill never tells an executor to fire a run, and that the
variant matches the deep-audit pipeline it wraps field by field apart from
the scout additions. 441 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The multi-model item was half-built: the catalog, the models action, and
the pricing/capability pre-flight all shipped, but a run still had exactly
one model for all six lenses. That is the wrong granularity. A stronger
model changes what the security and defect lenses find far more than it
changes an architecture map, so the useful knob is per lens, not per run.

lens_models in config.yaml routes individual lenses to their own batch
model. Each distinct model is resolved and pre-flighted independently:
priced from the live catalog, refused without structured-output support,
and clamped to its own completion ceiling. The estimate carries a model and
its rates per lens plus a mixedModels flag; the Pi confirmation names the
overridden model on the affected rows, so a mixed-model run cannot be
approved without seeing which lens costs what. Submission now fires from
the priced rows rather than recomputing them — the request that goes out is
the one the user approved. run-meta.json records lens_models, and collect's
truncation retry re-submits on the lens's own model and ceiling rather than
the run default, which would otherwise change the model mid-run.

An override naming an unknown lens id is dropped, not carried: it can only
be a typo, and a key that looks applied but is not is worse than one that
never appears.

What this deliberately does NOT do is change the default. The roadmap's
remaining item was "a stronger default for semantic lenses," and that is a
comparative-evaluation question on real repositories with real spend —
picking one here would spend every user's money on our guess. The config
says so and points at the models action. Per-model prompt tweaks stay open
for the same reason: they need the same evidence.

Tests: override parsing including the dropped-typo case, and an end-to-end
run asserting the override reaches the estimate with its own rates, both
levels of the batch payload, and the recorded per-lens entry, while the
other lens stays on the default. The config-key drift test now separates
top-level keys from nested examples instead of conflating them. 443 pass.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant