Skip to content

fix: a day-1 fieldbook shows no invented rows, and template prose stops passing for evidence - #47

Merged
suboss87 merged 2 commits into
Mainfrom
devin/1787800422-fieldbook-legibility
Aug 27, 2026
Merged

fix: a day-1 fieldbook shows no invented rows, and template prose stops passing for evidence#47
suboss87 merged 2 commits into
Mainfrom
devin/1787800422-fieldbook-legibility

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Opening .fde/ on day one showed rows that looked like captured facts but were placeholders — an assumption numbered 1, a risk with status open, a stakeholder whose stance was champion / neutral / resistant, an operating-map row, an eval golden G1. An FDE can't tell a seeded row from a real one, and a sponsor reading over their shoulder certainly can't. This deletes them and keeps the allowed values as a one-line legend under each table, so a table body is empty until someone puts something in it. parseMdTable already tolerates a header-only table. decisions.md's 9-line commented-out entry template moved to docs/schema.md under Decision entries, leaving a one-line example in place.

The interesting part is what re-rendering a placeholder as prose did to the doctor gates. Both defects below are introduced-and-fixed inside this PR — neither was reachable on Main — but both are the same latent class: a scanner that treats template text as evidence.

1. A legend satisfied the gate it documents. stripTemplateNoise() strips HTML comments and italic parentheticals only. The old cell was *(n/a | evals.md pass + HITL owner)* — invisible. The new **Eval receipt:** … else evals.md pass + the HITL owner. is not, so hasEvalReceipt() matched it and the AI in scope but no eval receipt rule stopped firing at ship and close — even with an explicit Verdict: NO-SHIP. Fixed at the rule, not the wording, since **Audit receipt:** was one keyword from the same failure:

function stripLegendLines(md) {   // a receipt is a dated line or a table row, never a bold label
  return md.split('\n').filter(l => !/^\s*\*\*[^*]+:\*\*/.test(l)).join('\n')
}
const del = stripLegendLines(stripTemplateNoise(readClean(eng, 'delivery.md')))

Applied to the delivery scans in hasEvalReceipt and hasValueBucket — the same rule hasOperatingMapContent already used. Both real receipt routes (a dated ## Ship receipts row, an evals.md **Verdict:** SHIP) still clear the gate.

2. Two counters read markdown emphasis as data.

if (/^[-*]/.test(t)) { n++ }               // countOpenRisks: "**Status:** `open`" → 1 open risk
if (/^[-*]\s/.test(t)) { n++ }             // fixed: the bullet marker must be followed by space

hasDatedWork = /\[\d{4}-\d{2}-\d{2}\]/.test(readEng('decisions.md') + )      // a dated example
hasDatedWork = /\[\d{4}-\d{2}-\d{2}\]/.test(stripTemplateNoise())           //   in a comment is not work

Either one flips fresh = !hasDatedWork && phaseUnset && !s.openRisks in collectDoctorIssues, so a brand-new engagement got nagged about success.md before the FDE had written a word — the exact wallpaper the day-1 gate exists to prevent.

Three regression tests: the shipped delivery.md cannot satisfy the eval gate while a dated receipt still can; a legend line counts as 0 open risks while fde log risk counts as 1; day-1 silence holds.

npm run check: 110 pass / 0 fail. No schema change, no new file, no dependency; .fde/ filenames, headings and required sections untouched.

Link to Devin session: https://app.devin.ai/sessions/f135381c4682413bae73dff38eb6d1a3
Requested by: @suboss87


Open in Devin Review

… open risk

Co-Authored-By: Subash Natarajan <suboss87@gmail.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

…gate requiring one

Co-Authored-By: Subash Natarajan <suboss87@gmail.com>
@devin-ai-integration devin-ai-integration Bot changed the title fix: a day-1 fieldbook shows no invented rows, and a legend is not an open risk fix: a day-1 fieldbook shows no invented rows, and template prose stops passing for evidence Aug 27, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

Runtime verification — day-1 fieldbook legibility

Adversarial matrix in 129 disposable HOME sandboxes, every invocation under timeout, with parent contrast in an 8b19ef7 worktree. Verified at b6c8002; the one regression it found is fixed in 9fe279d and re-verified.

Day-1 legibility, all 19 readers, the risk parser and the silence gate pass. One regression was found and fixed: the new delivery.md legend line silently disabled the ship/close eval-receipt gate.

🔴 Found at b6c8002: AI in scope but no eval receipt stopped firing at ship AND close — fixed in 9fe279d

stripTemplateNoise() strips only HTML comments and italic parentheticals. The old placeholder *(n/a | evals.md pass + HITL owner)* was invisible to the scanners; the new bold legend on templates/.fde/delivery.md was not, so it satisfied the gate it documents.

A doctor-rule sweep across all six phases, diffed against the parent, showed this was the only behavioural change, and it fired twice:

rules PARENT=20  HEAD=18
< [ship]  phase is ship with AI in scope but no eval receipt (…)
< [close] phase is close with AI in scope but no eval receipt (…)

The gate was dead even when the eval pack said NO-SHIP:

phase=ship, AI in scope evals.md absent present, unfilled Verdict: NO-SHIP
8b19ef7 (parent) blocked blocked blocked
b6c8002 not blocked not blocked not blocked
9fe279d (fix) blocked blocked blocked
H=$(mktemp -d); ROOT=$H/fde-engagements
HOME=$H FDEOPS_ENGAGEMENTS_ROOT=$ROOT node bin/fde.js resume --init aico
printf '\n## AI policy\nCustomer approved an LLM-assisted reranker; no PHI may reach the model.\n' >> $ROOT/aico/.fde/trust-profile.md
export HOME=$H FDEOPS_ENGAGEMENTS_ROOT=$ROOT FDEOPS_ENGAGEMENT=aico
node bin/fde.js log phase ship && node bin/fde.js doctor   # 4 issues at b6c8002, 5 at 8b19ef7 and at 9fe279d

Four wordings were verified to restore it (delete the line, wrap in *( … )*, drop the word pass, move into a comment) — but the fix landed at the rule instead, since **Audit receipt:** was one keyword from the same failure: stripLegendLines() drops **Label:** lines before the delivery scans in hasEvalReceipt/hasValueBucket, the rule hasOperatingMapContent already applied. Both real receipt routes still clear the gate, and a regression test now asserts the shipped template cannot.

✅ Day-1 legibility and the emptied tables

Data rows = 0 across all six tables (counting pipe-lines that are neither separator nor header). Zero open/champion/CRITICAL/G1/cost-save tokens inside any data row. No long-form decision scaffold left. All 19 verbs (resume, resume --full, triage, status, status --all, dashboard, doctor, prep, receipts, garden, capture, preserve, scan, demo, debrief, debrief --smart, ingest stage/propose/apply) run with no hang, no stack trace, no undefined/NaN. prep prints (none yet - log contacts with --signal) rather than a stakeholder named champion; the dashboard renders 0 phantom <tr> when empty and the real row when filled; ingest propose writes 0 bytes to .fde/ before apply.

The terrain "≥1 real exception row" rule — the one most suspected — is fully intact: silent at land/discover, fires at plan/build/ship/close on day 1 (identical to parent), satisfied by a real row, not satisfied by legend or prose alone. no value bucket still fires at ship: the **Bucket:** legend does not satisfy it.

Instruction density, measured honestly (a **Label:** with an empty value is a fill-in field, not an instruction):

file non-blank headings comments legends fields table hdr instruction %
decisions.md 5 3 2 0 0 0 100%
stakeholders.md 14 2 7 2 1 2 79%
delivery.md 17 6 3 4 0 4 76%
assumptions.md 7 1 1 3 0 2 71%
risks.md 5 1 1 1 0 2 60%
terrain.md 16 3 3 0 6 4 38%
countOpenRisks: 14/14 adversarial cases
case human HEAD PARENT
shipped day-1 template 0 0 0
*emphasis* / ***+--- rules / - stubs 0 0 2 / 2 / 2
**Status:** open · legend as shipped 0 0 1 / 1
2 real bullets · indented · unicode+CJK 2 2 2
escaped pipes | in cell 1 1 1 1
3 open + 4 under ## Retired 3 3 3
rows with empty cell #1 0 0 0
10k-line register 9999 9999 (58 ms) 10000

One framing note, now corrected in the description: the day-1 nag was never reachable on Main — its day-1 is already silent, because the legend line that triggers the miscount is added by this same PR. The trust.js fix is a necessary companion to the template change, not a fix for a shipped defect. Proven load-bearing with a hybrid build: NEW-tpl+OLD-codeopen risks:1 + nag; both consistent states → silent.

✅ Regression floor

Hostile FDEOPS_ENGAGEMENT (client-zzz, .., ../.., ./x, a/b, ., ???, ---, encoded traversal) → all exit 2, decoy manifests byte-identical, needle in 0 files. fifo/symlink/dir × all 5 memory slots (15 combos) → no hang, each correctly reported, symlink target body and <private> text leaked 0×; writes into a hostile slot refused exit 1 with the target sha unchanged. <private> needle + 2 fragments = 0 across 16 CLI surfaces, all 3 hooks, .debrief-propose and both 94 KB dashboards, with the needle confirmed on disk as a control. All .fde at <root>/<slug>/.fde; no real ~/fde-engagements created.

npm run check exit 0, npm test 110/110 at 9fe279d.

@suboss87
suboss87 merged commit 67c3b62 into Main Aug 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant