Skip to content

Task-file format checks are brittle: escalate PROMPT/STATUS that fail deterministic parse to an LLM normalization pass (or relax the checks) #618

Description

@HenryLach

Summary

The deterministic parsing of task files (PROMPT.md / STATUS.md) is strict enough that reasonable hand-authored variance causes severe failures rather than graceful handling. The strictness looks arbitrary relative to the fact that the ultimate consumers are LLM agents that readily infer intent from varied formats. Either the checks should be relaxed, or a failing file should be escalated to a non-deterministic (LLM) normalization pass instead of hard-failing.

Problem

Task files are hand-authored (by humans and by agents). A deterministic parser that hard-fails on benign format variance is fragile at exactly the wrong layer — the downstream worker/reviewer agents routinely handle far messier, more varied instructions than a strict ### Step N: grammar or a bulleted ## Dependencies block. So the format gate rejects inputs the actual executors would have understood, and the failure modes are severe:

All three are the same root shape: a strict deterministic format gate producing a severe outcome (false success, silent no-op, or batch crash) on benign hand-authored variance.

Proposed directions (pick one, or the hybrid)

Option A — Escalate to non-deterministic normalization. When a PROMPT.md/STATUS.md fails the deterministic checks, don't hard-fail — hand the file to an LLM pass (e.g. the supervisor agent) that reformats it to the canonical standard (### Step headings, a parseable ## Dependencies block, STATUS mirroring steps), then re-run the deterministic parse. Fail only if normalization can't produce a conforming doc.

Option B — Relax the deterministic checks. Make the parser tolerant of common variance (h2 vs h3 step headings; Step N — vs Step N:; deps as prose vs bullets; etc.), extracting only what it strictly needs and warning rather than failing on the rest. Since worker agents infer intent from varied formats, the parser can afford leniency.

Option C — Hybrid (recommended). Lenient deterministic extraction for the few scheduling-critical fields (ordered steps for progress tracking; dependency task-IDs for graph building); fail-soft with warnings instead of crashes/false-success; and escalate genuinely ambiguous files to the LLM normalization pass (Option A) before giving up. This keeps determinism where it materially matters while removing the brittle hard-fail cliffs.

Rationale

The consumers of these files are LLM agents that tolerate format variance well. A hard deterministic gate in front of them is a mismatch: it manufactures P0/P1 failure modes (#614, #615, #617) out of cosmetic differences. The format standard is still worth encouraging (canonical docs are easier to review), but conformance should be normalized-toward, not crashed-on.

Related

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions