feat(tasks): task-name variants, UGMathBench as _fixed, and grading off the shared event loop - #68

Merged

ethan-scitix merged 30 commits into

mainfrom

worktree-ugmathbench

Aug 8, 2026

Commits on Aug 8, 2026

feat(tasks): add UGMathBench (16 subjects, 3 randomized versions, EAcc)

ethan-scitix
and
claude
committed
b28297e
feat(tasks): task-name variants, and ship UGMathBench as _fixed (stable)

ethan-scitix
and
claude
committed
1759457
docs(tasks): the variant table is a vocabulary, not the whole mechanism

ethan-scitix
and
claude
committed
0293d4c
fix(ugmathbench): re-measure the _fixed divergence against upstream's judge

ethan-scitix
and
claude
committed
e905c3b
docs(ugmathbench): record the first live run; criterion 4 fails, stays experimental

ethan-scitix
and
claude
committed
701297d
fix(ugmathbench): parse the gold as sympy source; equivalence by substitution

ethan-scitix
and
claude
committed
d7335e5
feat(ugmathbench): promote to stable; audit criterion 4 against upstream's judge

ethan-scitix
and
claude
committed
995e218
fix(ugmathbench): guard the parser against model input; stop EAcc over-counting

ethan-scitix
and
claude
committed
dbc56b7
perf(core): grade in a worker process so one task stops stalling the session

ethan-scitix
and
claude
committed
7d33da4
refactor(tasks): use the house to_thread pattern where a thread actually works

ethan-scitix
and
claude
committed
dfc46f4
refactor(tasks): take every math-verify grader off the event loop, not just one

ethan-scitix
and
claude
committed
41f693e
fix(imo-bench): the fourteenth math-verify grader was still on the event loop

ethan-scitix
and
claude
committed
f04d97d
docs(core): the test-requirement commands no longer work as written

ethan-scitix
and
claude
committed
0963475
fix(mutmut): mutants/ was never an importable package, so no mutant ever ran

ethan-scitix
and
claude
committed
673fb7f
fix(mutmut): revert a scoping value I left in, and correct the reason it fails

ethan-scitix
and
claude
committed
4680874
test(core): measure the offload module's mutation score, and fix how it is run

ethan-scitix
and
claude
committed
2f94d25
test(core): raise offload's mutation score 51.6% -> 76.3% by asserting its contracts

ethan-scitix
and
claude
committed
8527f4a
fix(tests): assert the pool's construction instead of CPython internals

ethan-scitix
and
claude
committed
1d179f4
feat(scripts): enforce the mutmut config that made mutation testing unrunnable

ethan-scitix
and
claude
committed
cd11088
fix(ugmathbench): close five review findings from PR #68

ethan-scitix
and
claude
committed
35b696f
docs(scicode): the offload comment no longer describes every offload

ethan-scitix
and
claude
committed
e2f4995
fix(tasks): bound the two DeepSeek-Math graders in a worker process
ethan-scitix
committed
748890f
docs(core): record why offload cannot use anyio.to_process
ethan-scitix
committed
268bb1d
fix(tasks): close the last three review findings, and trim the prose

ethan-scitix
and
claude
committed
bfdc82d
fix(tasks): score an ungradeable answer wrong in the two DeepSeek-Math tasks

ethan-scitix
and
claude
committed
22b7719
fix(core): drop the dead offload pool, not just the flag that says it is dead

ethan-scitix
and
claude
committed
9890003
docs: narrow two claims to what the code actually does

ethan-scitix
and
claude
committed
2a972dd
fix(mutmut): the mutants copy still omitted scripts/, so stats never collected

ethan-scitix
and
claude
committed
a66e0af
docs(core): cut the mutmut troubleshooting transcript down to the rule

ethan-scitix
and
claude
committed
42e2e82
docs(datasets): stop specifying a patch-table format nothing has built yet

ethan-scitix
and
claude
committed
0fabdcb

Navigation Menu

Uh oh!

feat(tasks): task-name variants, UGMathBench as _fixed, and grading off the shared event loop - #68

feat(tasks): task-name variants, UGMathBench as _fixed, and grading off the shared event loop#68
ethan-scitix merged 30 commits into
mainfrom
worktree-ugmathbench

Commits on Aug 8, 2026

feat(tasks): add UGMathBench (16 subjects, 3 randomized versions, EAcc)

feat(tasks): task-name variants, and ship UGMathBench as _fixed (stable)

docs(tasks): the variant table is a vocabulary, not the whole mechanism

fix(ugmathbench): re-measure the _fixed divergence against upstream's judge

docs(ugmathbench): record the first live run; criterion 4 fails, stays experimental

fix(ugmathbench): parse the gold as sympy source; equivalence by substitution

feat(ugmathbench): promote to stable; audit criterion 4 against upstream's judge

fix(ugmathbench): guard the parser against model input; stop EAcc over-counting

perf(core): grade in a worker process so one task stops stalling the session

refactor(tasks): use the house to_thread pattern where a thread actually works

refactor(tasks): take every math-verify grader off the event loop, not just one

fix(imo-bench): the fourteenth math-verify grader was still on the event loop

docs(core): the test-requirement commands no longer work as written

fix(mutmut): mutants/ was never an importable package, so no mutant ever ran

fix(mutmut): revert a scoping value I left in, and correct the reason it fails

test(core): measure the offload module's mutation score, and fix how it is run

test(core): raise offload's mutation score 51.6% -> 76.3% by asserting its contracts

fix(tests): assert the pool's construction instead of CPython internals

feat(scripts): enforce the mutmut config that made mutation testing unrunnable

fix(ugmathbench): close five review findings from PR #68

docs(scicode): the offload comment no longer describes every offload

fix(tasks): bound the two DeepSeek-Math graders in a worker process

docs(core): record why offload cannot use anyio.to_process

fix(tasks): close the last three review findings, and trim the prose

fix(tasks): score an ungradeable answer wrong in the two DeepSeek-Math tasks

fix(core): drop the dead offload pool, not just the flag that says it is dead

docs: narrow two claims to what the code actually does

fix(mutmut): the mutants copy still omitted scripts/, so stats never collected

docs(core): cut the mutmut troubleshooting transcript down to the rule

docs(datasets): stop specifying a patch-table format nothing has built yet