Skip to content

OMARS generator: n_runs means the returned run count, and selection criteria search the design class (#496-#499) - #524

Merged
kgdunn merged 4 commits into
mainfrom
claude/process-improve-library-issues-hr7xxk-omars
Aug 29, 2026
Merged

OMARS generator: n_runs means the returned run count, and selection criteria search the design class (#496-#499)#524
kgdunn merged 4 commits into
mainfrom
claude/process-improve-library-issues-hr7xxk-omars

Conversation

@kgdunn

@kgdunn kgdunn commented Aug 28, 2026

Copy link
Copy Markdown
Owner

Summary

The OMARS generator cluster from the 2026-08-14 audit batch. Version bumped to 1.72.0 (MINOR: the n_runs semantics change plus the exact-search behaviour).

Every cell of all three issue tables was verified to reproduce the exhaustive optimum exactly (A: 13 cells, D: 14 cells, max|r|: 14 cells), including the former worst cases (A gap 39.7% at k=3 N=21; max|r| gap 51.2% at k=4 N=25). Wall-clock for the largest cell (k=4, N=25, h=12: 2.42M feasible designs) is about 90 s; the k=3 cells and k=4 up to N=17 are sub-5 s.

Test plan

  • New tests/test_omars_ilp_optima.py pins the tractable issue-table cells to their exhaustive optima (k=4 N=19/21 marked slow); the N=23/25 cells were verified in-session and left out of CI for runtime
  • New parity/sizing tests: returned row count equals n_runs for several (n_runs, center_runs) combinations, including an even total
  • uv run pytest tests/test_omars_ilp.py tests/test_omars_ilp_optima.py --no-cov: 63 passed
  • uv run pytest tests/test_omars*.py tests/test_experiments_omars.py tests/test_design_generation.py tests/test_audit_regressions_experiments.py --no-cov: 361 passed, 1 skipped (pre-existing skip)
  • ruff check ., ruff format --check . clean; mypy src/process_improve shows only the pre-existing mcp_server.py FastMCP attr error unrelated to this diff

Checklist

  • Version bumped in pyproject.toml (PATCH for fixes/docs/config, MINOR for new features)
  • Tests added or updated where relevant
  • ruff check . passes
  • CHANGELOG.md updated

@codecov

codecov Bot commented Aug 28, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 89.85507% with 21 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...c/process_improve/experiments/designs_omars_ilp.py 89.85% 12 Missing and 9 partials ⚠️

📢 Thoughts on this report? Let us know!

claude added 2 commits August 29, 2026 00:16
… criteria exact where enumerable

n_runs now counts the centre runs (issue #496): the design returned for
generate_omars(n_runs=n, center_runs=c) has exactly n rows, and
n_runs - center_runs must be a positive even number.

The a_optimal, d_efficiency, and min_second_order_correlation criteria
(and the D axis of the default dominance rule) previously post-hoc ranked
whatever the binary randomized-objective ILP multistart surfaced, which
cannot even represent designs that repeat a half-run; the exhaustive
optima in issues #497, #498 and #499 need such repetition. The search now
enumerates every feasible half-design multiset (counts per sign class)
up to four factors at moderate sizes, scores the count vectors in
vectorised batches without materialising designs, and picks the winner
exactly; larger problems keep the multistart, reported honestly as
search_mode="multistart". Regression tests pin all tractable cells of
the three issue tables to their exhaustive optima.

A design with a constant second-order column now scores inf on the
selection correlation metric instead of having the column skipped, so
min_second_order_correlation no longer favours degenerate designs.

Fixes #496. Fixes #497. Fixes #498. Fixes #499.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019C3XXbJkuSYH9fMryLqNcU
The typecheck gate is red on main itself: the unpinned mcp extra now
resolves to mcp 2.x, where FastMCP was renamed MCPServer, so mypy fails
on mcp_server.py on every branch. This ports the migrated server and the
mcp>=2.0 floor verbatim from #528; the diff no-ops once #528 merges.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019C3XXbJkuSYH9fMryLqNcU

kgdunn commented Aug 29, 2026

Copy link
Copy Markdown
Owner Author

The typecheck failure here is not this PR's: it is red on main's own latest runs. The unpinned mcp extra now resolves to mcp 2.x, where FastMCP was renamed MCPServer, so mypy fails on src/process_improve/mcp_server.py everywhere. The fix is PR #528; its migrated mcp_server.py and mcp>=2.0 floor are ported verbatim into this branch so CI can go green in any merge order. The ported files disappear from this diff once #528 merges.


Generated by Claude Code

@kgdunn
kgdunn merged commit 4b04915 into main Aug 29, 2026
14 checks passed
@kgdunn
kgdunn deleted the claude/process-improve-library-issues-hr7xxk-omars branch August 29, 2026 06:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment