Skip to content

Fix forge bench-mode helper imports from workspace root - #90

Open
irvineoy wants to merge 1 commit into
mainfrom
codex/fix-forge-benchmode-helper-path
Open

Fix forge bench-mode helper imports from workspace root#90
irvineoy wants to merge 1 commit into
mainfrom
codex/fix-forge-benchmode-helper-path

Conversation

@irvineoy

@irvineoy irvineoy commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • prepend the materialized task runner directory to Python module search paths in the canonical vLLM adapter block
  • include that bootstrap when materializing the canonical block into copied workspaces
  • add a no-GPU regression test for the exact launcher shape: a root-level driver loading scripts/task_runner.py with spec_from_file_location

Problem

PR #77 moved graph benchmark helpers from code inlined into task_runner.py to a sibling scripts/_aka_benchmark.py module. Task-provided forge drivers are copied to the workspace root, then load scripts/task_runner.py by file path. Python does not add the loaded file directory to sys.path, so the sibling helper cannot be found and forge --bench-mode exits with ModuleNotFoundError. The source-tree fallback also fails because an isolated run workspace does not contain the Arena src package.

This fixes the shared materialized adapter rather than patching individual task drivers, so existing and future affected tasks receive the same path bootstrap.

Verification

  • 22 passed: tests/test_perf_helper_materialization.py
  • 43 passed: tests/test_forge_kb_producer.py
  • 31 passed: tests/test_graph_benchmark_regressions.py and tests/test_benchmark_method_consistency.py
  • python src/tools/sync_perf_helpers.py --check reports all 426 benchmark entrypoints in sync
  • regression test runs without GPU and confirms a file-loaded runner resolves its sibling materialized helper from the workspace-root invocation shape

GPU validation was not run locally; CI or an MI355X smoke run should exercise the full forge --bench-mode path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant