Skip to content

eval: prepare explicitly gated PR80 annotation and review flow - #81

Merged
Air000000 merged 1 commit into
mainfrom
codex/refusal-prospective-train-run
Oct 16, 2026
Merged

Air000000 merged 1 commit into
mainfrom
codex/refusal-prospective-train-run

Conversation

@Air000000

Copy link
Copy Markdown
Owner

Scope

Prepare the first stage of merged PR80, without executing it. Reuse the existing frozen durable annotator in an isolated namespace; no new dependency/client/checkpoint framework, prompt, schema or dataset change. Bind the50 full-Top14 inputs, source/protocol/prompt/request fingerprints and future execution contract. Explicit PR80 data/model/budget approval must be obtained before contract preparation or paid execution; a merge-only/continue reply is not sufficient. The CLI confirmation flag records already-obtained approval and does not grant permission by itself.

Behavior

  • Up to50 annotation calls/CNY10 only, no paid retries; preserve existing exclusive lock, raw-before-parse, saved-response recovery, usage/accounting and unknown-charge stop behavior.
  • A schema-failed row cannot be paid again through a resume-after-audit switch. Only an audited saved-response citation recovery retaining the original class/source IDs can advance it. Raw journal remains unchanged.
  • Export all50 accounted drafts, validate complete question/draft/review order and cited lines, then freeze reviewed AI-proxy labels/counts/hashes before classification. Any class changes are recorded; AMBIGUOUS is not forced binary; no replacement to meet five/class.
  • Refuse incomplete annotation, existing classifier checkpoint or overwrite of frozen review/targets. Private draft/review/target rows stay under ignored data/; a future public freeze is aggregate hashes/counts only.
  • No classifier runner or predictions added in this PR. Same-family AI-proxy/not-independent-gold/no-promotion limits and total PR80 caps remain unchanged.

Current execution status

The first real-call command was rejected by automatic safety review before process creation because explicit approval for the new data/model scope was not established. It was not retried or bypassed. Verified absent: annotation execution contract/checkpoint, target freeze, target rows and classifier checkpoint. Actual provider calls0, actual new cost0. Paid work waits for explicit authorization of both PR80 stages (Singapore qwen3.6-plus-2026-04-02 annotation<=50, then frozen reviewed labels before qwen3.5-plus-2026-04-20 paired classification<=100; total CNY25, stage caps10/15, no DEV/recharge/paid retries).

Verification

  • 322 offline tests passed with external-network guard and temporary DB.
  • Simulated resume/recovery/no-paid-retry, unknown charge, locks, budget/call caps, authorization-before-inputs, tamper and preprediction review-freeze tests passed.
  • Frozen actual input/request/source bindings verified; Ruff, in-memory compilation and diff checks passed.
  • Primary dirty checkout/.env files/runtime data and all historical experiment artifacts untouched. Credential presence was checked without displaying secrets; no real request executed.

@Air000000
Air000000 merged commit 6c2a06b into main Oct 16, 2026
1 check passed
@Air000000
Air000000 deleted the codex/refusal-prospective-train-run branch October 16, 2026 13:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant