Skip to content

feat(eval): add isolated ROCm sanitizer plugins to main - #89

Merged
irvineoy merged 7 commits into
mainfrom
codex/pr78-to-main
Aug 27, 2026
Merged

feat(eval): add isolated ROCm sanitizer plugins to main#89
irvineoy merged 7 commits into
mainfrom
codex/pr78-to-main

Conversation

@irvineoy

Copy link
Copy Markdown
Collaborator

Summary

This PR brings the isolated ROCm sanitizer/evaluation-tool implementation from #78 onto main.

PR #78 was merged into its stacked base, codex/quality-loop-agent, rather than main. Its parent PR #76 remains an open draft, so the sanitizer implementation never reached the default branch.

This branch starts from the current origin/main and cherry-picks only the six commits belonging to #78, with -x provenance. It intentionally excludes the stacked branch's unrelated curated-task and quality-loop commits.

Conflict resolution

The first cherry-pick was reconciled with current main as follows:

  • kept both the current benchmark-method consistency fields and the sanitizer tool-gate fields in src/evaluator.py;
  • retained the evaluation-tool socket/artifact isolation in src/scripts/docker_benchmark.sh while excluding quality-loop-only mounts and commands;
  • excluded modifications to agents/quality_loop/*, because that agent is not present on main;
  • added only /.eval-tool-artifacts/ to .gitignore;
  • preserved current main runner cleanup while adding evaluation-tool temporary-path cleanup.

Validation

  • git diff --check origin/main...HEAD
  • Python compile checks for main.py, src/eval_tools, src/evaluator.py, and tests/eval_tools
  • Bash syntax checks for the Docker runner and its shell test
  • bash tests/test_docker_benchmark.sh
    • PASS: docker_benchmark runtime, agent-selection, and eval-tool isolation tests
  • Pinned scoring image, focused suite:
    • 193 passed in 5.01s
  • Pinned scoring image, full repository suite:
    • 459 passed, 4 skipped, 1 warning in 17.47s

Lineage

Original PR: #78

Original commits:

  • 844761cc feat(eval): add isolated ROCm sanitizer plugins
  • 25c3ddcd fix: harden evaluation tools fail-closed behavior
  • e8d594a5 docs: document planned TSAN and UBSAN support
  • 7ba42fe0 feat(eval): add Waitcheck and ConSan sidecars
  • ea0d0812 fix(eval): harden sanitizer startup controls
  • 6980af57 test(eval): require ConSan launcher provenance

@irvineoy
irvineoy merged commit e868efb into main Aug 27, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant