[STG-2284] fix(evals): default experimental to true on the SDK path

[miguelg719]

why

Running any agent eval through the verifier backend failed at V3 construction with ExperimentalNotConfiguredError — Feature "Agent callbacks" is an experimental feature, and cannot be configured when disableAPI: false.

initV3 defaulted experimental: false whenever a task passed no explicit value, but verifierAdapter installs agent callbacks (for trajectory recording) and hybrid agent mode also requires experimental. Core rejects both unless the instance is constructed with experimental: true. The verifier-carrier instance in benchHarness already sets it; the browser-driving instance did not.

what changed

One line in initV3.ts: when a task passes no explicit experimental value, default it to process.env.USE_API !== "true" instead of false.

This does not enable experimental in API mode. The default is gated on USE_API, so USE_API=true still resolves to experimental: false — identical to previous behavior. That gating is deliberate: core forbids the combination outright ("experimental mode cannot be used together with the Stagehand API"), so experimental is only turned on where disableAPI: true already holds, which is the pairing core requires.

This flag is unrelated to OTEL tracing. Traces reach LangSmith/Braintrust through the harness-level tracedSpan → OTEL exporter path, which never reads V3 config — verified by emitting a full nested task → agent.execute → verifier.* tree with no V3 instance at all. What this flag unblocks is whether a verifier-backed run can start in SDK mode; the two only looked coupled because a run that fails at construction produces no spans.

Pre-existing limitation this does not address: verifier-backed agent evals still cannot run in API mode, because core rejects experimental features whenever disableAPI: false regardless of the flag. Supporting that would need either core accepting agent callbacks in API mode, or the harness reconstructing the trajectory from the API response instead of callbacks.

test plan

• Unit suite green (392), typecheck + lint clean.
• Verified by running a real agent eval (b:onlineMind2Web, hybrid, gemini-3-flash) which previously failed at init for every task and now completes and scores.
• No change for USE_API=true runs (resolves to false, as before). Note: an end-to-end USE_API=true run has not been exercised on this branch — the reasoning above is from the code path, so worth a confirmation run before relying on API-mode behavior.

Stack 7/7 · base: #2371

[changeset-bot]

⚠️ No Changeset found

Latest commit: 3248fc3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

All reported issues were addressed across 1 file

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[kamath]

hell yeah

[miguelg719]

Superseded by the Stagehand 4.0 retarget stack (#2757–#2762). Main moved to the 4.0 monorepo restructure (packages/core → packages/sdk-ts; evals package restructured), so this 3.x-era diff no longer applies. The OTEL/LangSmith tracing work — plus native-span capture via 4.0's global tracer and claude_code/codex agent spans — is reimplemented additively on 4.0 in the new stack.