release: publish 0.24.4 — audit remediation port + durable review-verdict reuse - #794
Merged
Merged
Conversation
…e per-role tool/effort scoping Cherry-picked onto origin/main (the original branch drifted 10 commits). Conflicts were additive on both sides (main's --harness-only path + this advisor wiring) and were resolved by keeping both.
…no longer kills the heartbeat (AGT-4518) Cherry-picked onto origin/main. Verified main still has the defect: recoverPublishedRun compares payload_json (~runLedger.ts:1071) and reconcileDurableArtifacts has no per-row isolation.
…g past 200 (AGT-3421) Cherry-picked onto origin/main (main still has the fixed limit:200/offset:0).
…-CLI dropped-options warning (AGT-4444) Cherry-picked onto origin/main; main warns for mcpTools/coordinationContext only.
Main advanced 10 commits while `feat/omp-agent-patterns` was open, and its salvage PRs (#785-#791) fixed several of the same issues independently — its own way. Rather than rebase 10 commits through wide conflicts, this branch was recreated from `origin/main` and only the parts main does NOT already have were ported. Every ported item was checked against main first, with the evidence recorded, and where main's version was newer or better (memory pagination, the tester prompt bound, TUI width clipping, IPv6 callbacks, pipeline embed budgets) main's version was kept and nothing was ported. Still missing on main, now ported: - `shellCommandGuard.ts` — the guard is lexical on main (`BLOCKED_COMMANDS`), so 6 of 21 destructive forms execute (`r{m,} -rf`, `$'\x72\x6d' -rf`, `r"m" -rf`, `git clean -fdx`, …) while 4 of 51 harmless mentions are falsely blocked. Replaced with a bash-faithful resolver: 72/72 verdicts correct. - `read_file` — main reads the whole file before slicing; a 512 MiB file with `limit=1` throws `RangeError: Invalid string length`. Now a bounded window. - `recoverPublishedRun` idempotency + per-row reconcile isolation (AGT-4518): main still compares `payload_json`, so a recovery of an already-enqueued completion throws and kills the heartbeat. - `taskSource` pagination (AGT-3421): main still caps the local queue at 200. - Knowledge scanning incompleteness (AGT-3490): a walk stopped by depth, timeout or size still persists a graph indistinguishable from a complete one. - `gitInfo` discovery: main misses staged-but-uncommitted files entirely and newline-splits the log query, so a path containing a newline is corrupted. Also main strips a leading `\n` from EVERY filename token, though real `git log -z` prefixes only the first — verified against real git bytes. - String-aware JSON extraction for auditor/documenter (main is brace-blind), and the documenter prompt now delimits untrusted task/worker text. - Verification-evidence log bound: main joins every failing log before capping, so one suite's output displaces another's and the total is measured post-join. - Memory: LanceDB returns the vector column as an Arrow `Vector`, which main's `normalizeRecords` zeroes — embeddings are destroyed on rewrite and cosine similarity over stored rows is NaN, so dedup matched NOTHING. Confirmed against a live store. Plus the survivor refusal bound and non-quadratic consolidation (main's is still all-pairs under the global write lock). - Automation: `fixOne` bypasses the PR lease; decomposition capacity reads a PROCESS-LOCAL counter, so two real processes are both granted a slot against a cap of 1 (reproduced with real spawned processes). - Daily reporter: main republishes every already-succeeded project on retry. - Python task-state strictness (main's own test file is currently red: 5 failed), CI-wait duration validation, verify-manifest command/runtime caps, and the notifier delegating to the shared predicate — with TEST-NET-2/3 added so delegating does not regress what main's local table rejected. - CLI output bound, tester/dashboard hardening, Retry-After HTTP-date parsing where main still parses with `parseInt`, and the delegated-CLI warning now naming `protectedFiles`/`forbidPublication` in addition to the tool list. - `advisor` role + declarative per-role `tools`/`effort` (the harness patterns), which main does not have at all. Verification: `tsc` clean, `oxlint` clean, full suite 7167 passing. Each ported item carries a test that fails against `origin/main` and passes here; several were additionally mutation-checked (removing the bound/coercion/bucket key makes the named test fail).
…y (AGT-4597) `--concurrency` was a partition input on the audit path, not just a resource knob: `balanceAreasToConcurrency` re-split the source with a smaller per-area cap until `areas.length >= concurrency`, and `reviewMaxCommand` called it. So the same files and the same reviewer were judged in 2 units at concurrency 1 and 10 units at concurrency 8 — and since `aggregateAuditResults` is worst-wins (`decision: failed ? 'reject' : worst`), a finer split could only turn an approve into a revise/reject. Reproduced deterministically (20 files in one dir, cap 12, a reviewer that judges the unit it is handed): concurrency 1 -> 2 areas -> approve; concurrency 8 -> 10 areas -> reject. No LLM call, no timing. The feature that introduced it, INT-2249 / PR #183, records its own verification as "2 dirs, 10 files: concurrency 2 -> 2 areas, concurrency 8 -> 10 areas". The audit now partitions from `--max-files-per-area` alone via `planAuditAreas`; `--concurrency` only decides how many reviewers run at once. The capability INT-2249 wanted is kept, not dropped: the finer, more parallel fan-out is reached deterministically with `--max-files-per-area <n>` (cap 2 yields the same 10 areas the concurrency-8 path fabricated), now reproducible at any concurrency. The `--fix` path keeps `balanceAreasToConcurrency` unchanged — there is no verdict there and more areas just mean more parallel fix workers. Verified on the real CLI (`--dry-run` over this repo): 478 files -> 61 areas at concurrency 1, 8 and 32 alike; cap 4 -> 134 areas at 4 and 32. The regression test drives the real command entry and fails when the partition is re-derived from concurrency.
Promotes the Unreleased section to 0.24.4 and adds the durable review-verdict store: a review of content this deployment already reviewed is replayed from `(kind, base, content digest)` instead of paying for the reviewer again, and the store survives a daemon restart. Measured over the recorded history (530 records, 512 with a verdict): 68 byte-identical pairs, 15 same-mode, 10 repeating the same verdict — ~10 of 512 reviews, at ~$0.26 and 93s p50 per reviewer call. Proven across two separate processes: 27.0s / 6 model calls became a 1.58s replay with zero model calls, same verdict. The advisor pass still runs over a reused verdict, so a replay can tighten but never fail open. Also corrects the README's `review --max` claim that areas "auto-split to fill --concurrency" — the partition has been derived from --max-files-per-area alone since AGT-4597, so --concurrency no longer changes the verdict.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recreated from
origin/mainrather than rebasing. Main advanced 10 commits of remediation while the original branch was open (PRs #785/#786/#787/#788/#789/#791), and those fixed several of the same issues its own way — so a rebase would have produced wide conflicts over duplicated work. Instead this branch starts atorigin/mainand ports only the parts main does not have.Every item below was checked against main first; where main's version was newer or better (memory lifecycle pagination, the tester prompt bound, TUI width clipping, IPv6 PKCE callbacks, pipeline embed budgets,
changeStatusoldValue, the daily watermark retry,isValidationRelevantFile) main's version was kept and nothing ported.Still missing on main — now ported
Shell safety (AGT-3436/3486). Main's guard is lexical (
BLOCKED_COMMANDS), so 6 of 21 destructive forms execute (r{m,} -rf,$'\x72\x6d' -rf,r"m" -rf,git clean -fdx,rm -fr, an unterminated quote) while 4 of 51 harmless mentions are falsely blocked (# rm -rf,echo "rm -rf is blocked"). Replaced with a bash-faithful resolver (quote/backslash/$'…'decoding, comments, brace expansion,$IFS, substitutions, launcher and nested-shell following): 72/72 verdicts correct.read_filealso threwRangeError: Invalid string lengthon a 512 MiB file withlimit=1; now a bounded window.AGT-4518. Main still compares
payload_jsoninrecoverPublishedRun, so recovering a run that already enqueued its own completion throwsOutbox dedupe key collision— and with no per-row isolation that aborts the whole heartbeat. Repro test included.AGT-3421. Main's local queue is still capped at 200 with no pagination; later tasks are never selected.
AGT-3490. A scan stopped by depth, timeout or the 512 KiB size cap still persists a graph indistinguishable from a complete one. Now
incomplete/incompleteReasonsthrough the graph, schema (optional, so legacy snapshots load), snapshot andscanAndCache.gitInfo discovery. Main misses staged-but-uncommitted files entirely (
diff --cachedabsent) and newline-splits the log query, corrupting a path containing a newline. Verified against realgit log -zbytes: main also strips a leading\nfrom every filename token although only the first of each commit carries it.Parsing/evidence. Auditor/documenter counted braces without tracking strings or escapes (an unmatched brace or escaped quote loses the structured fields); the documenter prompt interpolated task/worker text raw; verification evidence joined every failing log before capping, so one suite's output displaced another's.
Memory. LanceDB returns the vector column as an Arrow
Vector, which main'snormalizeRecordszeroes — so embeddings are destroyed on rewrite and cosine similarity over stored rows is NaN, meaning dedup matched nothing. Confirmed against a live store (ctor=Vector, isArray=false, cosine=NaN). Plus the survivor refusal bound and non-quadratic consolidation (main's is still all-pairs under the global write lock).Automation.
fixOnebypasses the cross-process PR lease. Decomposition capacity reserves under a lock but reads a process-local counter, so two real spawned processes are both granted a slot against a cap of 1 — reproduced.Others. Daily reporter republishes already-succeeded projects on retry; Python task-state strictness (main's own test file is currently red: 5 failed); CI-wait duration validation (
NaNmade the poll sleep 0 ms); verify-manifest command/runtime caps; notifier delegating to the shared predicate with TEST-NET-2/3 added so delegating does not regress what main's local table rejected; CLI output bound; Retry-After HTTP-date where main stillparseInts; the delegated-CLI warning now namingprotectedFiles/forbidPublication.Harness patterns.
advisorrole (second opinion that can only tighten the gate, fail-open) and declarative per-roletools/effort— main has neither.Verification
tscclean ·oxlintclean · full suite 7167 passing.Each ported item carries a test that fails against
origin/mainand passes here (several were run against a scratch copy of main to prove it). Several were additionally mutation-checked: removing the survivor bound, the Arrow coercion, the consolidation bucket key, or the per-page dedup each makes the named test fail.Also raised from 30 s to an explicit 120 s the three
src/verify/runner.test.tscases that build a real git sandbox — they pass in ~0.3–3 s alone but exceeded the global timeout under full-suite load, which is a guard that only fires when the whole suite runs.Added: the reported defect — the audit verdict depended on
--concurrency(AGT-4597)openswarm review --max --concurrency <N>returned a different verdict for the same file set depending on N.--concurrencywas not just a resource knob:balanceAreasToConcurrencyre-split the source with a smaller per-area cap untilareas.length >= concurrency, andreviewMaxCommandcalled it on the audit path. SinceaggregateAuditResultsis worst-wins (decision: failed ? 'reject' : worst), a finer split could only turn anapproveinto arevise/reject.Reproduced deterministically — 20 files in one directory at cap 12, with a reviewer that judges the unit it is handed (what a real reviewer subagent does with an area prompt). No LLM call, no timing:
--concurrencysrc/big (1/2)[12],src/big (2/2)[8]The feature that introduced it records its own verification: INT-2249 / PR #183 — "2 dirs, 10 files: concurrency 2 to 2 areas, concurrency 8 to 10 areas". Pool saturation was bought by changing the units of judgement.
Two further layers found: with a rate limit the skipped areas are
total - attempted, so a higher concurrency left more source unreviewed (1/2 attempted at concurrency 1 vs 1/10 at concurrency 8 — the samerejecton different evidence); andisInfraErrormatches the pool's ownskipped:sentinel because the message embeds its cause text (true), harmless today only becauseif (infraAbort) return;precedes the streak increment — recorded as latent, not changed.Fix: the audit partitions from
--max-files-per-areaalone via a newplanAuditAreas;--concurrencydecides only how many reviewers run at once. The capability is kept, not dropped — the finer fan-out is reached deterministically with--max-files-per-area <n>(cap 2 yields the same 10 areas the concurrency-8 path fabricated), now reproducible at any concurrency. The--fixpath keepsbalanceAreasToConcurrencyunchanged.Verified on the real CLI (
--dry-runover this repo, 478 files): 61 areas at concurrency 1, 8 and 32 alike; cap 4 yields 134 areas at concurrency 4 and 32. The regression test drives the real command entry and fails when the partition is re-derived from concurrency (demonstrated at both the command call sites and the planner, then reverted).