Skip to content

docs: reconcile refusal overview with frozen v5.2–v5.3 evidence - #79

Merged
Air000000 merged 1 commit into
mainfrom
codex/refusal-results-docs-sync
Oct 16, 2026
Merged

Air000000 merged 1 commit into
mainfrom
codex/refusal-results-docs-sync

Conversation

@Air000000

@Air000000 Air000000 commented Oct 16, 2026 •

Copy link
Copy Markdown
Owner

Problem

After PR78 merged, the root README, evaluation overview, project summary and demo guide still described v5.2 as never model-evaluated/only synthetic. This confused the zero-call mechanism demo with the separately completed real-model experiments.

Changes

  • Correct those four existing entry points only; no new document/framework or interview/coaching material.
  • Link frozen v5.2 fictional controls, v5.2.1 contemporaneous format regression, natural v5.1/v5.2.1 diagnosis and v5.2.1/v5.3 observed regression.
  • Preserve the actual boundaries: controls are not natural accuracy; repeated controls did not demonstrate a format fix; latest binary proxy agreement22/25→23/25, two refusal fixes/one new refusal, same insufficient admission1/4; five ambiguous targets excluded from binary scoring.
  • Explain that the latest contemporaneous baseline22/25 must not be replaced by historical21/25. AI proxy/observed development is not independent expert validation, generated-answer quality or online evidence.
  • Close further prompt iteration/repeated paid calls on these same30 under the completed scope. Future new-sample validation must define/freeze scope and obtain separate data/cost authority.

Verification

  • 313 offline tests passed with external-network guard and temporary DB.
  • Checked all80 local Markdown links in the four documents; latest figures agree with the frozen public result JSON.
  • Existing zero-call synthetic demo passed; Ruff and diff checks passed.
  • Four Markdown files only,49 insertions/9 deletions; no runtime, dependency, prompt, validator, dataset, frozen report/contract or CI edits.
  • No provider calls, new fees, DEV, relabeling or serving changes. Primary dirty checkout/.env/runtime data untouched.

@Air000000
Air000000 merged commit ce222f8 into main Oct 16, 2026
1 check passed
@Air000000
Air000000 deleted the codex/refusal-results-docs-sync branch October 16, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant