hostcdp.sh drives every rep from one process, as the VM arm does

[ejc3]

The host-container control (hostcdp.sh) started a timing wrapper (python3 -) and then python3 cdpdrive.py for every request, so each host wall_ms included two interpreter start-ups. The VM arm never pays that: reqbench.py imports cdpdrive once and times one drive() call per request (reqbench.py:3172, :3270). The render benchmark page charges 60.7 ms of the host container's wall time to this wrapper.

Contract: the host control times one in-process cdpdrive.drive() per request. Records keep their fields, and its refusals keep their exit codes. run.json says which way a record was driven.

Downstream impact: host-control records produced after this merge, which are not comparable to earlier ones on wall_ms. No VM-arm code changes.

Changes

bench/chromium/hostcdp.sh

• One python3 - process imports cdpdrive.py and runs every rep, timing each drive() on the monotonic clock. It passes the arguments cdpdrive.py ADDRESS URL --format jpeg --nav-timing parsed to.
• Each record is written before its rep is judged. A non-numeric or unreadable 1-minute load refuses with exit 5, and a failed rep exits 4, as before.
• The driver field holds the whole result JSON. The old loop kept only its last 2,000 characters, which compare.py then parses as JSON.
• run.json gains "driver_process": "in-process". Records without it started a process per rep.
• The loop is a heredoc in hostcdp.sh, which is hashed before and after every run. The sealed source lists and runtime manifests are unchanged.

Tests

• test_hostcdp_corpus.write_python_shim logs every python3 start and hands the in-process loop a stub cdpdrive module. test_hostcdp and the CPU-budget tests use it.
• HostCdpDrivesInProcess.test_no_rep_starts_an_interpreter fails on main: reps ran cdpdrive.py as their own program, and python3 starts grew with REPS.
• test_corpus_mem's HostCdpProducer harness moves its per-rep driver actions (load-file changes, runtime tamper, a leftover descendant) into the stub's drive().
• test_numeric_output_from_a_failed_load_read_is_invalid is removed along with its cut shim branch. It covered cut exiting 9 while printing a number, and the per-rep load read no longer runs cut. The start-of-run gate still does, and keeps its status=9 test.
• The run.json metadata test pins driver_process and fails without it.

Evidence

$ python3 -m unittest discover -s bench/chromium -p 'test_*.py'
Ran 1053 tests in 263.886s
OK

The measured rerun of the host control is part of the pbox campaign that follows this PR. This box's load was 34 from other work, above the control's 1.0 gate.

Summary by CodeRabbit

• Benchmarking
  • Chromium benchmark repetitions now run through a single driver process.
  • Each repetition records its result, elapsed time, URL, warmup status, and load reading before evaluation. Invalid load readings, driver failures, and interruptions are recorded as benchmark errors.
  • Benchmark metadata and comparison reports show the driver process mode. Older runs without this metadata are labeled as using per-repetition subprocesses.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-02T02:40:58.751808Z	8b2778a	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

🧰 Additional context used

📚 Code guidelines (1)

bench/chromium/AGENTS.md — auto-discovered

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 6097200f-5de8-4a41-875d-2847c6e03cce

📥 Commits

Reviewing files that changed from the base of the PR and between 8b2778a and 6fa0b27.

📒 Files selected for processing (4)

• bench/chromium/compare.py
• bench/chromium/hostcdp.sh
• bench/chromium/test_corpus_mem.py
• bench/chromium/test_hostcdp_corpus.py

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The host CDP harness now runs scheduled repetitions through one Python process that imports the driver once. It records each repetition’s result and load reading. Tests use a shared Python shim and check that interpreter starts do not increase with repetition count.

Changes

Host CDP driver execution

Layer / File(s)	Summary
In-process driver loop bench/chromium/hostcdp.sh, bench/chromium/compare.py	The harness records driver_process as "in-process", imports cdpdrive once, and calls drive() for each scheduled URL. It writes and flushes each result before validating the load reading and driver outcome. Exit status 4 remains a request failure; other nonzero statuses exit as refusal status 5. Comparison output reports the recorded driver process or defaults to "per-rep subprocess" for older records.
In-process driver test coverage bench/chromium/test_hostcdp_corpus.py, bench/chromium/test_hostcdp.py, bench/chromium/test_corpus_mem.py	A shared Python shim supports the in-process invocation and logs interpreter starts. Tests check repetition counts, standalone driver launches, interrupts, and invalid results. Corpus fixtures update driver, load-reading, and metadata expectations.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Sequence Diagram(s)

sequenceDiagram
  participant HostHarness as hostcdp.sh
  participant PythonRunner as Python runner
  participant Cdpdrive as cdpdrive
  HostHarness->>PythonRunner: Start in-process driver loop
  PythonRunner->>Cdpdrive: Import driver once
  loop Each scheduled URL
    PythonRunner->>Cdpdrive: Call drive() and time the repetition
    Cdpdrive-->>PythonRunner: Return driver result
    PythonRunner->>HostHarness: Write and flush repetition record
  end
  PythonRunner-->>HostHarness: Exit with driver or measurement status

Loading

Merge Risk: ⚪ Minimal · up to 6fa0b

No identified issue blocks the in-process driver change from merging after normal checks.

Security Architecture Review

Security architecture risk: 🔵 Low · up to 6fa0b

The reviewed paths preserve fresh CDP sessions, run ownership checks, and withdrawal of failed runs. No new security boundary bypass was established. Broader deployment exposure remains unverified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

• inferred — The inspected input-to-sink path remains the benchmark’s configured URL flowing into Chromium Page.navigate, followed by screenshot decoding and result-file writes. Process reuse does not add a new caller or credential source in this path; the broader accessibility of the host-network browser was not verified.

Trust Boundaries and Controls

• observed — Existing controls bind rows to run metadata, validate repetition order and URLs, reject withdrawn or incomplete datasets, and recheck producer-source identity before summary generation. These checks remain between driver-controlled results and accepted comparison output.

Resilience and Maintainability Implications

• observed — Per-call socket cleanup covers normal and exceptional paths. Existing run-directory claiming, owner-checked container cleanup, and failure withdrawal provide containment across repetition, interruption, partial output, and concurrent invocation. Successful completion is published only after the supervised lifecycle succeeds.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 27.78% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 18 functions across 5 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: hostcdp.sh now drives every repetition from one process, matching the VM arm.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}