Skip to content

fix: one bad record no longer fails the whole set (C5 rows 8–12, #1251) - #1411

Merged
Juliusolsson05 merged 16 commits into
mainfrom
fix/c5-fail-all-batch
Sep 27, 2026
Merged

Juliusolsson05 merged 16 commits into
mainfrom
fix/c5-fail-all-batch

Conversation

@Juliusolsson05

@Juliusolsson05 Juliusolsson05 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Refs #1251 (row 13 moved to #1417, which closes the issue when it merges; row 10 is strict by design) (rows 8–12 of the C5 hunt; row 13 moved to #1417, because it and steering q109 there change the same ledger loader; rows 14–15 are strict by design and were only recorded there).

One bad record must cost only itself. Every row was verified on origin/main 5e22c7b0, and every new test was run red against main's implementation first. Plan and per-row evidence: docs/plans/2026-09-27-c5-fail-all-batch.md.

Row Bug on main Fix Test (red on main)
8 One unreadable Codex rollout (EACCES, EIO) made readline rethrow through fromHead, rejecting discover() and emptying the Codex column Skip that rollout without caching it, so it is retried later codex.system.test.ts: EACCES
9 One unparseable incident row hid the run's whole incident list, and a helper restart in that run rewrote incidents.json from an empty list, erasing the readable evidence and the unknown row Parse per row; carry unknown rows verbatim through every rewrite; mark the store degraded MonitorHistoryStore.test.ts: null
10 Malformed Pi line blocks switch, duplicate and rewind No change: strict by design. It feeds destructive transforms, and skipping a line would move a conversation with a silent hole none
11 One invalid approval entry made every authorize() throw, blocking all repository workflows Skip the entry (it approves nothing, so it is prompted again) and carry it through persist(). A wrong file version still throws WorkflowSourceApprovalStore.test.ts
12 One bad identity failed the whole TLDR/goal/enforcement batch, blanking Agent Activity The shape stays strict. Invalid identities are dropped, which is exact because the store only writes valid ones new tldr/ipc.test.ts: ZodError

The owner's "do not delete stuff often" rule shaped rows 9 and 11: records this build cannot read are carried, never dropped.

Mutations (each applied alone, then restored):

  • row 9, drop the re-append: red;
  • row 9, drop the degraded flag: red;
  • row 11, drop the carry: red;
  • row 11, let an unreadable entry approve: red.

Verification:

  • npx tsc -b: 0 lines.
  • Touched suites: codex 7/7, MonitorHistoryStore 7/7, approvals 2/2, tldr 94 across src/main/tldr, debugRetention 9/9.
  • Note: the untouched "lists the same rows for a family whose roots keep their case" test hit its 5 s timeout once while the machine was loaded by other suites. It passed on the immediate rerun. I'm reporting it, not widening it.

Review round 1 (all FIX-BEFORE-MERGE), fixed at 7204503c:

  • A wholly refused incident file is set aside to incidents.refused-<ms>.json, never overwritten, and its run is never expired.
  • A carried-row shortfall sets shortened.
  • Codex index rows are typed by value, so a BLOB title no longer rejects the index.
  • Skipped rows and rollouts are counted into the span, lastDowngradeReason and a warning.
  • TLDR/goal history returns an empty list for an invalid identity.
  • An approval without approvedAt prompts.

Each fix is fail-first or has its mutant red. Details are in the plan's review table.

Residuals:

  • Row 9: carried foreign rows never expire on their own. They leave only with their run directory, and a run whose file holds only foreign rows is not expired-by-emptiness (it is still pruned by the data budget).
  • Row 12: an invalid identity gets no record and no error (history is empty), the same answer as "no TLDR yet".
  • Row 8: the picker has no degraded indicator for any source yet, including the existing no-index downgrade. Skips are logged, not shown.

🤖 Generated with Claude Code

Juliusolsson05 and others added 6 commits September 27, 2026 04:27
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e list (#1251 row 8)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ugh rewrites (#1251 row 9)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rkflow (#1251 row 11)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng it (#1251 row 12)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…runing (#1251 row 13)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05 Juliusolsson05 added the sev:P3 Minor label Sep 27, 2026
@Juliusolsson05 Juliusolsson05 added type:bug Something works wrong class:C5-fail-all One bad record takes everything down labels Sep 27, 2026
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
…runing (#1251 row 13)

Moved here from #1411 (manager q109: coordinate the overlap). Row 13 and
steering q109 both change loadManualLegacyBundlePaths, so they ship together.
A JSON-valid non-entry line (null, a number, a non-string bundlePath) threw,
which rejected collectArtifacts and stopped every prune pass. Rows are now
shape-checked; a non-string reason counts as manual, so the bundle is kept.
Adds a test through the real loader and cache: #1411 review (b) found that
replacing the loader's parse with an empty set survived the parser-only test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
…codex

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 and others added 6 commits September 27, 2026 05:19
Row 13 of #1251 and steering q109 on #1417 both change
loadManualLegacyBundlePaths, so the row ships with #1417 (commit 02df3652
there), with a test through the real loader. This PR no longer touches
debugRetention.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…overwritten or expired (#1411 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ort skipped rows and rollouts (#1411 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…review survivor)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05

Juliusolsson05 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner Author

Review disposition (round 1: a, b, c codex at 129f8c5a, all FIX-BEFORE-MERGE)

Fixed at 7204503c. Per-finding detail is in docs/plans/2026-09-27-c5-fail-all-batch.md (review table).

Finding Verdict Disposition
a1 / b1 / c1, major: a wholly refused incident file was overwritten by the next same-run write, and its prior run was expired valid Fixed. The file is set aside (incidents.refused-<ms>.json, bytes kept and budgeted) before the first write, nothing is written if that fails, and maintenance keeps the run. Three fail-first cases.
a / c survivor: the foreign-only expiry guard valid Pinned: removing the foreign or the refused guard goes red.
b2, major: a BLOB title rejected the whole Codex index valid Fixed: rows are typed by value and the title falls back. Fail-first on the recorded corpus.
c2, minor: a skipped rollout looked complete valid Fixed (logged): a count reaches the span, lastDowngradeReason and a warning. The picker indicator is a stated residual (no source has one).
b3, minor: history threw for an invalid identity valid Fixed: empty history. Mutant red.
a3, minor: carried rows silently displaced a new incident valid Fixed: shortened is set. Fail-first.
a survivor: the approvedAt check valid Pinned. Mutant red.
b survivor: the row 13 loader valid Moved to #1417 with row 13 (manager q109). Tested through the real loader there.
a2, minor: carried rows reorder declined Every value is kept, and no reader gives order any authority.

Round-2 verification is queued under the reviewer cap.

Round 2 (a, b, c codex at 7204503c): all FIX-BEFORE-MERGE, the final round

The round-1 reproductions are all verified fixed, and every round-1 survivor is now killed. Fixed at 56891812:

Finding Verdict Disposition
a / b / c, major: a later run's retention deleted a set-aside refused file valid Fixed: runs holding an incidents.refused-* file are never expired (found at startup and on set-aside). Fail-first.
a / b / c, major: a same-millisecond second refusal overwrote the first set-aside file valid Fixed: a UUID in the name. Fail-first with a fixed clock.
c survivor: a stale refusal marker after set-aside valid Pinned (records again after the set-aside).
b survivor: the rename-failure guard valid Pinned (a read-only run dir: the file is kept byte-for-byte and shortened is set).
a / c survivors: the column type guards valid Pinned: every projected string column is BLOBed, plus a fallback-chain row. Four mutants red.
c3, minor: the picker shows a partial list as complete residual Filed as #1433.

Steering q115 (held before the member gate): fixed at 3c807a1b

A failed per-run startup listing was read as "no set-aside file" (.catch(() => [])), so maintenance could delete a prior run holding only refused evidence. Fixed: a failed listing marks the run unindexed (never expired) and degrades the store; only ENOENT means nothing to find.

  • Test: a real temp run holding only incidents.refused-*, with one injected listing failure at startup. Red at 56891812 (the run was deleted); removing the mark is red.
  • Loss-path audit (in the plan): whole runs leave disk only via clear(), budget pruning, and marker-guarded expiry.

Both review rounds are spent. The head needs a manager-verify (a, b, c) of the round-2 fixes and q115 before the member gate.

Manager-verified a / b / c (temp/review-1411/manager-verify-*.md) at 3c807a1b: the round-2 fixes and q115. merge-gate --member --dry: GATE PASS (3c807a1b, 0 behind main, checks green). READY.

Residuals, not blocking:

Juliusolsson05 and others added 4 commits September 27, 2026 07:02
…llides (#1411 review round 2)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pe (#1411 review round 2 survivors)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… is kept (steering q115)

Startup found set-aside refused files by listing each run with
.catch(() => []). A failed listing read as 'no set-aside file', so a prior
run holding only one had no marker and the next maintenance deleted it with
the refused bytes (the unknown-as-empty shape q109 forbade). Now the run is
marked unindexed (never expired) and the store degraded; only ENOENT means
nothing to find. Real temp run + one injected per-run listing failure: red at
5689181 (ENOENT on the deleted run); removing the unindexed mark is red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05

Copy link
Copy Markdown
Owner Author

B6: changed 'Closes #1251' to 'Refs #1251'. Row 13 moved to #1417, which has not merged yet, so merging this PR alone must not close #1251. #1417 carries the closing reference.

@Juliusolsson05 Juliusolsson05 changed the title fix: one bad record no longer fails the whole set (C5 rows 8–13, #1251) fix: one bad record no longer fails the whole set (C5 rows 8–12, #1251) Sep 27, 2026
@Juliusolsson05
Juliusolsson05 merged commit 80d014a into main Sep 27, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the fix/c5-fail-all-batch branch September 27, 2026 19:07
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
…ed-run

Keeps both sets of unknown-is-protected rules: main's refused / foreign
incident runs and set-aside scan, and this branch's examinedRuns,
touchedSince and foreign-file fingerprints; main's rewrite path in
writeRunIncidents is behind foreignChanged too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
…sted alone

After merging main, touchedSince kept #1411's fresh 1970-clock fixtures on
its own, so the foreignIncidents, refusedAsideRuns and listing-failure
unindexed guards lost their failing tests. Those tests now use the real
clock and aged() fixtures; each fails without its guard. Adds the two-store
carried-rows retention test (B6 check 2110).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

class:C5-fail-all One bad record takes everything down sev:P3 Minor type:bug Something works wrong

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant