fix(cli): distinguish degraded judge/synthesis runs from a clean pass (DSE-901) - #60
Merged
Conversation
… (DSE-901) Adversarial (and synthesize/debate) mode exited 0 even when the judge/ synthesizer call failed while members still answered -- a verification gate reading only the exit code (e.g. /conclave-verify) could not tell a partial run from a clean pass. This reproduced live on 2026-07-25: an Anthropic credit failure took out the claude judge while 4/5 members answered, adversarial.verdict came back null, and the CLI exited 0. CouncilResult gains a computed `degraded` field (True when synthesis_error or adversarial.verdict_error is set), serialized as a top-level JSON key. The CLI now exits a distinct code 3 for a degraded run, keeping the existing 0 (clean)/1 (hard failure: zero usable answers or Elite not ready)/2 (usage error) meanings unchanged. Judge/synthesizer failover to another provider is explicitly out of scope for this fix (follow-up).
ernestprovo23
added a commit
that referenced
this pull request
Aug 1, 2026
Pre-existing formatting drift (inline-comment spacing) unrelated to any specific PR's diff — confirmed byte-identical to origin/main before this fix, so every PR touching this repo inherits a failing ruff-format check regardless of what it actually changes. Surfaced while merging PR #60 (DSE-901).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
adversarial(andsynthesize/debate) mode exited0even when the judge/synthesizer call failed while members still answered. A verification gate that only checks the exit code (e.g./conclave-verify) reads a partial run as a full pass. Reproduced live on 2026-07-25: an Anthropic credit failure took out theclaudejudge, 4/5 members still answered,adversarial.verdictcame backnull, and the CLI exited0.CouncilResultgains a computeddegradedfield —Truewhensynthesis_errororadversarial.verdict_erroris non-None— serialized as a top-level key in every--jsonpayload. The CLI now exits a new, distinct code3for a degraded run. Existing exit codes are unchanged:0= clean pass,1= hard failure (zero usable member answers, or Elite notready),2= usage/config error.3was chosen (not the ticket's example of2) specifically to avoid colliding with the existing usage-error meaning of2.DOCUMENTATION_INDEX.md's CLI-tests row,tests/test_cli.py's module docstring, and aCHANGELOG.mdentry. No full contract rewrite — DSE-752 owns that separately.Files changed
src/conclave/models.py—CouncilResult.degraded(Pydanticcomputed_field)src/conclave/cli.py—_DEGRADED_EXIT_CODE = 3; wired into the--json, human-render, and--streamexit paths (in that priority order: hard failure > degraded > clean)tests/test_cli.py,tests/test_synthesizer.py— new tests (see below)CHANGELOG.md,DOCUMENTATION_INDEX.md— exit-code contract docsRelease classification
~/.claude/scripts/release_control.py classifyreturned:{"classification": "security-specific", "reasons": ["security-control behavior changed"], "classifier_version": "dse-release-control-v1"}Do not merge without a human receipt (per
rules/release-control.md).Manual read of the reasons: the deterministic keyword scanner matched generic English words in this diff's docstrings/tests —
"role"(fromif m.get("role") == "system", a message-dict key, not an authz role),"scope"(from "out of scope for this fix"), and"verification"(from "verification-gate consumer"). No auth/session/credential/IAM/tenant-isolation/CORS/rate-limit code path is touched. That said, per the binding rule the classifier's output is authoritative over agent judgment, so this is flagged as-is rather than self-cleared — a human should confirm before merge.Test plan
pytest— full suite: 733 passed, 91.17% coverage (CI gate is 75%)ruff check .— all checks passedruff format --check— the 4 touched Python files are formatted (README.md has a pre-existing, unrelated formatting drift not touched by this PR)tests/test_cli.py:test_adversarial_judge_failure_json_is_degraded_not_clean,test_adversarial_judge_failure_human_exits_degraded_code(mock the judge call to raise, mirroring the live incident),test_adversarial_clean_pass_still_exits_zero(regression: full success still exits 0),test_all_members_failed_synthesize_mode_exits_one_not_degraded(regression: hard failure still exits 1 even thoughdegradedalso readsTruethere — priority order verified)tests/test_synthesizer.py: unit pins for thedegradedcomputed field (test_degraded_false_by_default,test_degraded_true_when_synthesis_error_set,test_degraded_true_when_only_adversarial_verdict_error_set,test_degraded_false_on_clean_synthesis,test_degraded_is_a_top_level_json_key), plusassert result.degraded is Trueadded to the existing judge-failure/unkeyed-judge/synthesizer-failure tests🤖 Generated with Claude Code