Skip to content

feat(cli): multi-agent developer experience - #1272

Merged
sebyx07 merged 4 commits into
mainfrom
feat/multi-agent-dx
Nov 19, 2026
Merged

sebyx07 merged 4 commits into
mainfrom
feat/multi-agent-dx

Conversation

@sebyx07

@sebyx07 sebyx07 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

An auditor used Ultimate the way parallel agents do: two clones of one app, merged branches, one shared machine. It found 12 problems. This PR fixes all 12, each with a test that failed first. Four workers wrote the fixes and one integrator merged them. Additive only: no breaking change ships on main.

The 12 findings

# Severity Finding Fix
1 critical After two branches that each ran x db gen were merged, the gate suggested x db gen 'record alphas'. That migration could never replay (42P07). New X_MIGRATION_FORKED (@ultimat3/db, migration-fork*.ts). x db gen, x db migrate and the drift step check for it first. x db gen --merge-fork corrects only the newest .snapshot.json. It changes no SQL and deletes nothing. When both branches create the same object, the fix says to run x db status --json on every database first.
2 high Two parallel x g resource branches conflicted in 7 files. A union merge produced files that do not parse. x merge-driver (install / registry / catalog). bun run setup installs it (now 7 steps). .gitattributes marks which driver each file uses. New apps write one registry entry per line. Existing apps get the change from the merge-drivers codemod. x doctor warns X_MERGE_DRIVER_UNSET. Many-Agents has a new "Merging two features" section.
3 high A conflict marker in en.json crashed x doctor with a fix that pointed back at the doctor. x manifest blamed 23 modules instead of the catalog. X_MERGE_UNRESOLVED, X_CATALOG_UNPARSEABLE (names the file), and X_DOCTOR_PROBE_FAILED: a doctor check that throws becomes a finding instead of a crash.
4 high Fix lines ended in then: x verify. That command is not found, and if it ran it would start the whole gate. runnableFix now rewrites then: x / then x, and so do the stderr step pointer and x errors explain. 24 shipped guards now end with their own step (x verify --only boundaries). New scripts/fix-tails.ts (X_FIX_TAIL_BARE, X_FIX_TAIL_WHOLE_RUN) holds the line.
5 high Three fixes sent a worker to a full run. X_TEST_NO_FILES suggests x test --affected --dirty. X_TYPECHECK_FAILED names the file to edit and x verify --only typecheck. An unknown command whose nearest match is test or verify suggests --help instead of the bare command.
6 high A ledger wait named the holder but not what to do, and a wait had no limit. Each wait line ends "leave it running; it starts by itself". x heavy run --max-wait exits with X_HEAVY_WAIT_EXPIRED (75). x heavy status shows the wrapper and command pids (wrapperAlive, childPids). "admission in progress" replaces -. A run also waits while the CPU is saturated and another run is admitted.
7 high The scaffolded agent instructions contradicted the many-agents rules. Scaffold CLAUDE.md lists the worker commands. /feature and /verify skip the gate when the worker has a file set. .claude/settings.json allows bunx x heavy and bunx x stop. Existing apps get this from the docs-worker-commands codemod.
8 medium A neighbour's dev server turned x doctor red, and the suggested port could already be taken. freeDevPortPair returns a pair that is free when asked. The holder's directory is in the cause. A port held by another checkout is a doctor warning, wired into data.warnings. The sync role's X_PORT_IN_USE uses the same free-pair search.
9 medium X_JOB_UNREGISTERED hid the job's file. It carries the module path, sets at, and prints the complete import line.
10 medium Two x db gen runs at the same time in one checkout put one agent's table in the other's migration. A per-checkout db-gen lock. The success line lists the tables. "No migration needed" names the migration that already records them.
11 low 2 of 4 of 6 workers was hard to read. It now reads ran 2 at once (peak), planned 3, or 1 worker, one per file (plan: …).
12 low Test service stacks were invisible, never stopped, and shared by app name. One services: started/reused/shared <project> line. x clean --services stops the stack. A stack with the same name from a different app is refused with X_TEST_SERVICES_UNAVAILABLE.

Fixed while integrating

  • Orphans. If bun run preflight or bun run coverage is stopped by a signal, it now kills every process it started (scripts/lib/stop-tree.ts, tagged through ULTIMATE_VERIFY_STEP). Before, the child coverage gate and its bun test workers kept running and kept the ledger reservation they inherited. A test reproduces the orphan and proves the fix.
  • Serve graph pin (989 → 992). The new fork check came in through @ultimat3/db's barrel, so every role loaded three modules it never uses. It moved to @ultimat3/db/schema-dump, and serve-graph.test.ts now lists the fork modules as never loaded at boot. The one remaining module, port-pair-free.ts (the sync role's free-pair fix), raises SERVING_ROLE_CEILING 1547 → 1548 and WEB_ROLE_CEILING 1747 → 1748. That is the measured amount, and the reason is in each fixture.
  • DoctorProbe stays additive. portOwner, mergeState and mergeDriverUnset are optional, so scripts/lib/public-api.lock.json records no break.
  • Ratchets back at their pins: fix-prose (verify-services.ts), test typecheck (2 errors), test bare error, a fix line in a test that could not run, a mkdtemp in a test that runs git (now tempTree), and a name that tripped X_TEST_PORT_PROBED (the CLI helper is now freeDevPortPair).
  • Brief codemods that failed quietly (found upgrading six real apps to 31.48.0). In 4 of 5 apps, docs-many-agents reported changed: [], skipped: [] when the conventions table had been reworded or lived in AGENTS.md. In one app it pushed CLAUDE.md to 12,071 bytes, past the 12,000-byte cap (X_AGENTS_MD_TOO_LARGE). It and docs-worker-commands now share codemods/agents-doc-row.ts. That helper looks in CLAUDE.md, then AGENTS.md. When it cannot place the row, it reports the file as skipped with the exact row to add. Before writing, it checks the size against the agentsMdMaxBytes budget and reports the byte count when the row would not fit. A test covers each case and failed on the old code.
  • x upgrade installs the merge drivers a hop's .gitattributes names, so upgrade-fixtures no longer counts X_MERGE_DRIVER_UNSET as a hand step.
  • In-process test fixtures no longer join the real ledger queue and leave frozen tickets at its head.
  • Also in this branch: the security-release devops schedule (thu/sun), and the tracked-app wording in scripts/help.ts and PUBLISHING.md.

Docs

Many-Agents (merging, --merge-fork, the x db gen lock), Migrations-And-Backfills, Testing, CI-Parallel-Gate, Tutorial-01, Upgrading (a row for the 24 guard fixes), the error-code pages, CHANGELOG [Unreleased], and plan 101 status.yml.

No release is cut here: the maintainer app releases this.

🤖 Generated with Claude Code

sebyx07 and others added 3 commits November 18, 2026 23:42
Work from four DX workers on the 12 findings of the multi-agent audit:
migration forks, merge drivers, runnable fix tails, worker-safe fixes,
explained ledger waits, scaffold instructions, free port pairs, job
registration paths, concurrent db gen wording, worker counts and test
service stacks. Integration follow-ups land in later commits.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- x doctor reports another checkout's port in data.warnings; the sync
  role's port refusal walks to a free pair (freeDevPortPair).
- preflight and the coverage gate kill the tree they started on a
  signal, so no orphan keeps its ledger reservation.
- migrationFork moves to @ultimat3/db/schema-dump and is pinned as
  never-at-boot; the serving and web ceilings rise by the one module
  the sync refusal needs.
- DoctorProbe's new members are optional, so the public API stays
  additive; the new codes and their fixes split out at the ceiling.
- Ratchets back at their pins; the setup-commands rule scopes a
  mention inside a table row to that row.
- Many-Agents states --merge-fork and the x db gen lock; CHANGELOG
  [Unreleased], an Upgrading row for 24 guard fixes, plan 101 status.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docs-many-agents and docs-worker-commands report a skip with the row
  to add when the table is reworded or missing, find it in AGENTS.md
  too, and skip with the byte count rather than push the brief over
  its agentsMdMaxBytes budget (agents-doc-row.ts).
- x upgrade installs the merge drivers a hop's .gitattributes names,
  so the upgrade fixtures owe no X_MERGE_DRIVER_UNSET hand step.
- dev-lock-free-port.ts becomes port-pair-free.ts: production reaches
  no dev-* module.
- migrationFork has one home, @ultimat3/db/schema-dump.
- Tests follow one registry entry per line and the setup driver step;
  file-scope reset hooks where a suite can skip; the repo-scan budget
  on fix-tails.test.ts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@developerz-ai

developerz-ai Bot commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Fix failing CI on #1272

Working on it.


- app-evals.test.ts writes its fixture apps under a fresh directory per
  pass: bun test --rerun-each keeps the module cache, so a second pass
  over the same paths registered no prompt.
- The three test names the vanished-tests rule missed are kept on the
  cases that replaced them.
- new-shapes.test.ts reads the dashboard's queries one entry per line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sebyx07
sebyx07 merged commit ddcbe02 into main Nov 19, 2026
47 checks passed
@sebyx07
sebyx07 deleted the feat/multi-agent-dx branch November 19, 2026 05:55
sebyx07 added a commit that referenced this pull request Nov 19, 2026
…1273)

Main went red on the merge of #1272 (ddcbe02) in two jobs that run
only on a push to main:

| Job | Cause | Fix |
|---|---|---|
| ci / deploy-proof | `docker/deploy-proof/held-job.sh` edited
`defineApi` lists written on one line; `x new` now writes one entry per
line | it inserts each module after the list's opening line; checked
against a fresh `x new proofapp` |
| tutorial-run | Tutorial 2 shows `✓ migration …_add_todos generated`;
`x db gen` now adds ` — tables: todos` | the page shows the new line;
`bun run scripts/tutorial-run.ts` ran green locally (20 steps) |

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
sebyx07 added a commit that referenced this pull request Nov 19, 2026
Two problems seen on 2026-11-19.

## 1. `x upgrade` always ran the full gate

On a loaded machine one app's hop took 55 minutes, with no way to hand
`unit`, `live` and `e2e` to CI.

| | |
|---|---|
| flag | `x upgrade --only <step>[,<step>…]`: the hop's last stage is `x
verify --only <steps> --json` instead of `x verify --json` |
| reader | `packages/cli/src/verify-only.ts`, the one `x verify --only`
now uses too: same step list, same declared order, same refusals
(`X_CLI_BAD_FLAG`, fix = the corrected `x upgrade --only …`) |
| honest report | `proof: "verify-only"`, `verify.only[]`,
`verify.notRun[]`, summary `… green on typecheck,lint only — NOT THE
FULL GATE; not run: …`, a human line `not run by this hop … x verify
--only <rest> --json`, and a `then` naming that command |
| default | unchanged: no flag, or every step named, is the full gate
and reports as one (`proof: "verify"`, no `only`/`notRun` keys) |
| no app | refused: the hop runs scripts there, not `x verify`, so a
list would narrow nothing |

The earlier stages (codemods, manifest, doctor, guards, a Bun-floor
hop's coverage rebaseline, exercise) run either way, because each one
writes what the hop leaves behind. No template changes, so no codemod
row. Docs: `wiki/CLI-Reference.md`, `wiki/Upgrading.md`, `CHANGELOG.md`.

## 2. Jobs that run only on `main` broke `main` after green PRs

| Job | On a pull request now |
|---|---|
| `ci / deploy-proof` | runs when `scripts/deploy-proof-due.ts` finds a
`VERDICT_PATHS` file in the PR diff (merge commit vs first parent,
checkout depth 2), and skips its heavy steps otherwise. Inherits
nothing; `main` still decides by the predecessor rule |
| `tutorial-run.yml` | `pull_request.paths` = `push.paths`. The list
gained `cmd-new*`, `cmd-db*`, `db-*`, `messages*` (#1272's break was `x
db gen` output). Concurrency is grouped per ref, so main never cancels a
PR's run |

- `verify` is still the one verdict: neither job is in its `needs` and
neither is required. A path-filtered workflow never leaves a check
pending.
- `scripts/ci-workflow-main-only.test.ts` enforces the rule. A workflow
pushed on `main` with no `pull_request` trigger, or with different
paths, fails the test unless it is a named publisher (`wiki.yml`,
`scorecard.yml`). So does a `ci.yml` job whose `if` admits a push but
not a PR.
- `scripts/lib/health-tutorials.ts` reads `--branch main` only, so a
PR's red tutorial run never counts against main's health.
- `docs/history/ci-split.md` has a new dated section.

## Checks run locally

- `bun test` on the upgrade, verify-only, deploy-proof-due, health,
tutorial-run and ci-workflow* suites passed.
- New tests fail against the old sources and workflows.
- `bun run typecheck` passed.
- `bun run verify --only
typecheck,lint,boundaries,filesize,package-shape,errors,manifest`
(through `x heavy run`) passed after the fixes.
- actionlint and zizmor found nothing new.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant