Goal
Make the public codewhale exec surface reproducible enough for a small future pull request adding Codewhale as a built-in Verifiers v1 harness.
Codewhale only needs to own its launch contract. Verifiers owns interception, traces, model-call timing, token accounting, retries, rollout limits, and runtime orchestration. Do not add a Verifiers runtime, trace parser, receipt schema, or integration package to Codewhale.
Do not submit the external PR or publish anything in this issue.
Baseline
- Codewhale: current
origin/main when assigned (contract audited at 6cd2953956fd69406aa126f3f58e631332d90abe; rebase before review)
- Verifiers:
v0.2.1, commit ab65b6e8d34b03d162408d4bcb854430a86809e6
Verifiers v1 built-in harnesses install a pinned CLI in setup(), launch it against the supplied interception endpoint and secret in launch(), and call runtime.run_program(...). Match that existing shape.
Codewhale changes
- Add public
--no-project-config support to the codewhale dispatcher and forward it before the TUI subcommand.
- Make
codewhale --no-project-config exec ... skip the workspace-specific user-config overlay that Exec currently applies unconditionally.
- Document and test one isolated headless launch using existing generic surfaces:
- an explicit temporary config with
[providers.openai] base_url, model, and api_key_env;
CODEWHALE_HOME set to a fresh per-run directory;
CODEWHALE_SECRET_BACKEND=file;
CODEWHALE_MCP_CONFIG pointing to a generated per-run MCP JSON file;
CODEWHALE_MEMORY=false and CODEWHALE_TELEMETRY=false;
CODEWHALE_ALLOW_INSECURE_HTTP=1 only when Verifiers supplies a trusted http:// interception endpoint (container/tunnel endpoints are not always loopback);
exec --auto --sandbox danger-full-access --output-format stream-json;
--append-system-prompt and --disallowed-tools when supplied by the caller.
The interception secret must remain in the child environment. The temporary route config stores only the environment-variable name, never the secret:
provider = "openai"
[providers.openai]
base_url = "<interception-endpoint>"
model = "<model-id>"
api_key_env = "VF_CODEWHALE_API_KEY"
The generated MCP file contains only the task servers supplied by the caller:
{"mcpServers":{"task-tools":{"url":"<mcp-url>"}}}
The exact argument order is:
codewhale \
--config .vf-codewhale/config.toml \
--workspace . \
--no-project-config \
--skip-onboarding \
exec \
--auto \
--sandbox danger-full-access \
--output-format stream-json \
-- "<prompt>"
Files
crates/cli/src/lib.rs — parse/forward --no-project-config and focused dispatcher tests
crates/tui/src/main.rs — honor the flag in Exec and focused unit tests
crates/tui/tests/verifiers_harness_contract.rs — provider-free launch/MCP acceptance
docs/AGENT_RUNTIME.md — concise public contract and future-upstream checklist
Do not change crates/tui/src/config.rs or crates/tui/src/mcp.rs unless the focused test finds a real defect; route-bound api_key_env, CODEWHALE_MCP_CONFIG, the mcpServers alias, and URL-based Streamable HTTP/SSE fallback already exist.
Acceptance
Verification
cargo test -p codewhale-cli --locked no_project_config
cargo test -p codewhale-tui --bin codewhale-tui --locked no_project_config
cargo test -p codewhale-tui --test verifiers_harness_contract --locked
cargo test -p codewhale-tui --all-features --locked
cargo fmt --all -- --check
cargo clippy --workspace --all-targets --all-features --locked -- \
-D warnings \
-A clippy::uninlined_format_args \
-A clippy::too_many_arguments \
-A clippy::unnecessary_map_or \
-A clippy::collapsible_if \
-A clippy::assertions_on_constants
git diff --check
Cloud worker handoff
Start from a fresh clone of Hmbown/CodeWhale; fetch current origin/main, create agent/4641-verifiers-ready, record the base SHA, and read AGENTS.md, CONTRIBUTING.md, and crates/tui/AGENTS.md. Use temporary homes, loopback fixtures, and placeholder secrets only. No machine-local handoff, existing worktree, installed Codewhale, external checkout, live credential, or paid resource is required.
Use signed-off commits (git commit -s) and open one focused PR to Codewhale main with exact provider-free receipts. Rebase before review, wait for hosted CI, and resolve every review thread. Stop at the tested PR; maintainers own independent audit, merge, installed dogfood, publication, and the later upstream submission.
Future upstream PR (after the public v0.9.1 artifact exists)
The expected Verifiers change is limited to:
verifiers/v1/harnesses/codewhale/__init__.py
verifiers/v1/harnesses/codewhale/harness.py
verifiers/v1/harnesses/__init__.py
tests/v1/conftest.py (add codewhale to the existing harness matrix)
pyproject.toml (register the strict pytest marker)
docs/v1/harnesses.md
CodewhaleHarnessConfig pins 0.9.1. setup() downloads the matching GitHub Release archive plus codewhale-bundles-sha256.txt, verifies the archive, and installs it into the runtime. launch() writes the temporary route/MCP files above, conditionally allows the Verifiers-owned insecure HTTP endpoint, and calls runtime.run_program(...). Set APPENDS_SYSTEM_PROMPT = True, SUPPORTS_MCP = True, and leave message prompts/user simulation unsupported.
Reuse Verifiers' existing tests/v1/test_e2e.py::test_agentic matrix; upstream guidance explicitly says not to add unit tests. Do not add a Codewhale-specific trace parser.
Holdouts and out of scope
- The upstream install path cannot be completed until a public, immutable Codewhale
v0.9.1 GitHub Release and checksum manifest exist.
- Opening/submitting the upstream PR and running its credentialed
PRIME_API_KEY E2E matrix require separate approval. No live provider, hosted-runtime, or paid credential is used here.
- Do not tag, publish, create a release, or mutate
PrimeIntellect-ai/verifiers in this issue.
- Do not claim runtime/architecture support until the exact released archive has run in that upstream runtime.
Goal
Make the public
codewhale execsurface reproducible enough for a small future pull request adding Codewhale as a built-in Verifiers v1 harness.Codewhale only needs to own its launch contract. Verifiers owns interception, traces, model-call timing, token accounting, retries, rollout limits, and runtime orchestration. Do not add a Verifiers runtime, trace parser, receipt schema, or integration package to Codewhale.
Do not submit the external PR or publish anything in this issue.
Baseline
origin/mainwhen assigned (contract audited at6cd2953956fd69406aa126f3f58e631332d90abe; rebase before review)v0.2.1, commitab65b6e8d34b03d162408d4bcb854430a86809e6Verifiers v1 built-in harnesses install a pinned CLI in
setup(), launch it against the supplied interception endpoint and secret inlaunch(), and callruntime.run_program(...). Match that existing shape.Codewhale changes
--no-project-configsupport to thecodewhaledispatcher and forward it before the TUI subcommand.codewhale --no-project-config exec ...skip the workspace-specific user-config overlay thatExeccurrently applies unconditionally.[providers.openai] base_url,model, andapi_key_env;CODEWHALE_HOMEset to a fresh per-run directory;CODEWHALE_SECRET_BACKEND=file;CODEWHALE_MCP_CONFIGpointing to a generated per-run MCP JSON file;CODEWHALE_MEMORY=falseandCODEWHALE_TELEMETRY=false;CODEWHALE_ALLOW_INSECURE_HTTP=1only when Verifiers supplies a trustedhttp://interception endpoint (container/tunnel endpoints are not always loopback);exec --auto --sandbox danger-full-access --output-format stream-json;--append-system-promptand--disallowed-toolswhen supplied by the caller.The interception secret must remain in the child environment. The temporary route config stores only the environment-variable name, never the secret:
The generated MCP file contains only the task servers supplied by the caller:
{"mcpServers":{"task-tools":{"url":"<mcp-url>"}}}The exact argument order is:
Files
crates/cli/src/lib.rs— parse/forward--no-project-configand focused dispatcher testscrates/tui/src/main.rs— honor the flag inExecand focused unit testscrates/tui/tests/verifiers_harness_contract.rs— provider-free launch/MCP acceptancedocs/AGENT_RUNTIME.md— concise public contract and future-upstream checklistDo not change
crates/tui/src/config.rsorcrates/tui/src/mcp.rsunless the focused test finds a real defect; route-boundapi_key_env,CODEWHALE_MCP_CONFIG, themcpServersalias, and URL-based Streamable HTTP/SSE fallback already exist.Acceptance
--no-project-config, forwards it beforeexec, and rejects misplaced usage clearly.Execdoes not apply a matching global[workspace]/[projects]overlay when that flag is present.CODEWHALE_HOMEand reaches the exact configured endpoint/model.Verification
Cloud worker handoff
Start from a fresh clone of
Hmbown/CodeWhale; fetch currentorigin/main, createagent/4641-verifiers-ready, record the base SHA, and readAGENTS.md,CONTRIBUTING.md, andcrates/tui/AGENTS.md. Use temporary homes, loopback fixtures, and placeholder secrets only. No machine-local handoff, existing worktree, installed Codewhale, external checkout, live credential, or paid resource is required.Use signed-off commits (
git commit -s) and open one focused PR to Codewhalemainwith exact provider-free receipts. Rebase before review, wait for hosted CI, and resolve every review thread. Stop at the tested PR; maintainers own independent audit, merge, installed dogfood, publication, and the later upstream submission.Future upstream PR (after the public v0.9.1 artifact exists)
The expected Verifiers change is limited to:
verifiers/v1/harnesses/codewhale/__init__.pyverifiers/v1/harnesses/codewhale/harness.pyverifiers/v1/harnesses/__init__.pytests/v1/conftest.py(addcodewhaleto the existing harness matrix)pyproject.toml(register the strict pytest marker)docs/v1/harnesses.mdCodewhaleHarnessConfigpins0.9.1.setup()downloads the matching GitHub Release archive pluscodewhale-bundles-sha256.txt, verifies the archive, and installs it into the runtime.launch()writes the temporary route/MCP files above, conditionally allows the Verifiers-owned insecure HTTP endpoint, and callsruntime.run_program(...). SetAPPENDS_SYSTEM_PROMPT = True,SUPPORTS_MCP = True, and leave message prompts/user simulation unsupported.Reuse Verifiers' existing
tests/v1/test_e2e.py::test_agenticmatrix; upstream guidance explicitly says not to add unit tests. Do not add a Codewhale-specific trace parser.Holdouts and out of scope
v0.9.1GitHub Release and checksum manifest exist.PRIME_API_KEYE2E matrix require separate approval. No live provider, hosted-runtime, or paid credential is used here.PrimeIntellect-ai/verifiersin this issue.