fix(providers): send a per-sample x-session-id so stateful routers see one task per sample

[ayush-or]

TL;DR

Inference requests now send x-session-id = <sessionId>.<epoch>.<sampleId> while everything else (result store, logs, progress, response-cache salt) keeps using the run-level sessionId, so openrouter/jev and the sticky-session pin no longer treat every sample in a run as a continuation of the first one.

What changed?

• New src/runtime/request-session-id.ts: a currentSampleIdRef FiberRef (set per sample in harness/run.ts, next to setCurrentEpoch) and buildRequestSessionId(sessionId, epoch, sampleId).
  • Format ${sessionId}.${epoch}.${sanitizedSampleId}; sample chars outside [A-Za-z0-9_-] become - so /, spaces, etc. cannot break the dot hierarchy.
  • Bounded to 256 chars (OpenRouter's SESSION_ID_MAX_LENGTH): an overlong sample segment is truncated and suffixed with a 12-hex sha256 so long ids stay unique. Falls back to the bare run id when epoch or sample is unknown, or when there is no room.
• responses-client.ts: header x-session-id now uses the per-sample id. buildResponseCacheSalt still receives config.sessionId, so resume/replay cache keys are unchanged. ResponsesConfig.sessionId semantics are unchanged.
• agent-cli/runner.ts: runAgentCli reads the same epoch and sample FiberRefs and nests them into ORI_OPENROUTER_SESSION_ID before exec. This covers every ori subprocess caller in one place (deep-swe, terminal-bench ori-solver, swe-atlas, and future callers) since those bypass the Responses client. Solver files are untouched. wandr/deep-swe checkpointKey values (/-separated) are checkpoint-store keys, not request ids, and are untouched.

Why?

Session id has routing priority over conversation id. With one id per child workflow, jev's decisions-router state carried the incumbent model across unrelated GPQA questions (390/396 requests stayed on the first pick), and the sticky endpoint/model-slug pin, Fortuna sampling key, and provider prompt-cache affinity were all run-scoped. Nesting epoch and sample under the run id gives each task fresh router state and keeps multi-turn continuity within a task, while openrouter-web's ClickHouse realized-cost and router-usage queries already match session_id = p OR startsWith(session_id, concat(p, '.')), so run-level reporting rolls up unchanged.

Slack thread: https://openrouter.slack.com/archives/C0BMHG5CG1E/p1789781836317759

Searched existing PRs (open, draft, merged, "session"); none matched.

How to test

bun test src/runtime/request-session-id.test.ts src/providers/responses-client.test.ts src/benchmarks/terminal-bench/ori-solver.test.ts

Expected: the client test asserts x-session-id: wf-123.2.q-17 alongside x-openrouter-cache-salt: wf-123:epoch-2 for sessionId=wf-123, epoch 2, sample q/17. The ori-solver test asserts ORI_OPENROUTER_SESSION_ID=run-1234.1.terminal_bench-adaptive-rejection-sampler for epoch 1.

Benchmark impact

Routing only, no scorer/solver/dataset change. Runs of openrouter/jev (and to a lesser degree auto, pareto, phaser) will pick a model fresh per sample instead of inheriting the run's first pick, for both Responses-client benchmarks and agent-CLI benchmarks. Endpoint selection is per sample rather than pinned per run. Response-cache replay for in-flight runs is unaffected because the salt is unchanged.

Follow-ups in openrouter-web (not in this PR): bench_run_no_prompt_cache.tf groups by exact @extra.session_id from provider-side logs and the monitor-benchmark-run SKILL uses session_id = '<workflow_id>'; both need a prefix match if they read the request id. Harness logs still emit the run-level id.

Reviewer focus

• buildRequestSessionId sanitization/truncation rules.
• agent-cli/runner.ts: the control-character guard still runs on the caller's opts.sessionId; the nested id only adds ., digits, and [A-Za-z0-9_-], so it cannot introduce one.

Checklist

• Tests cover changed behavior
• Public API or configuration changes are backward compatible, or the break is documented
• Benchmark changes document dataset provenance and licensing (n/a)
• No credentials, private results, or restricted dataset contents are included
• Documentation is updated where needed (n/a)

Link to Devin session: https://openrouter.devinenterprise.com/sessions/b91ee8adb7514c2fa3f15e8dfa1dfd1b
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/b91ee8adb7514c2fa3f15e8dfa1dfd1b?variant=devin
Requested by: @ayush-or

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'DevinAI' or '@devin'.

• Disable automatic comment, CI, and merge conflict monitoring

---

Original prompt from Ayush

SYSTEM:
<latest_message>
Ayush Patel (U0B8L6RNMA9) [ts=1789782133.457669]: @Devincan you scope out in breadth where we'd run into side effects if we changed the session id for benchmark temporal workflows in benchmark-harness

and reporting/queries in openrouter-web etc
</latest_message>

=== BEGIN THREAD HISTORY (in #eng-benchmarks) ===
Ayush Patel (U0B8L6RNMA9) [ts=1789781836.317759]: is the shared session id across benchmark samples intentional for routing too, or just tracking? feels like sessions should be per task for stateful routers, right?

today it's session_id = <child_workflow_id>
should it be session_id = <child_workflow_id>.<epoch>.<sample/task_id>

e.g. our gpqa run used one session id for all the different questions, so openrouter/jev treated them as continuations of the same task. 390/396 requests kept claude fable high instead of trying to picking a model fresh for each question due to a policy

Abhinav Pola (U090K0G7JF3) [ts=1789781914.673809]: It was meant to be for tracking but has unintended routing effects

Abhinav Pola (U090K0G7JF3) [ts=1789781957.335589]: We should prob use user id instead but it might affect some queries

Ayush Patel (U0B8L6RNMA9) [ts=1789782133.457669]: @Devincan you scope out in breadth where we'd run into side effects if we changed the session id for benchmark temporal workflows in benchmark-harness

and reporting/queries in openrouter-web etc
=== END THREAD HISTORY ===
Channel ID: C0BMHG5CG1E
Thread URL: https://openrouter.slack.com/archives/C0BMHG5CG1E/p1789781836317759?thread_ts=1789781836.317759&amp;cid=C0BMHG5CG1E

The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra ... (22 chars truncated...)

[devin-ai-integration]

Post-work validation for b2fb1a9 passed. Every benchmark sample sends a per-sample session id through both the Responses client and the ORI subprocess, and the ClickHouse parent predicate rolls them up without cross-run leakage.

Summary:

• Happy path tests: 7/7 passed
• Failure path tests: 5/5 passed
• Load/concurrency: 12 Responses requests plus 12 ORI execs at concurrency 3 and 2, all carrying the expected id
• Follow-ups (openrouter-web, not this PR): bench_run_no_prompt_cache.tf groups by exact @extra.session_id, and the monitor-benchmark-run skill queries session_id = '<workflow_id>'. Both need the startsWith(session_id, concat(p, '.')) form.

Method: a real runBenchmark run drove the production makeOpenRouterModelLayer and runAgentCli against a local capturing HTTP endpoint and a fake sandbox. No network, no real key. Harness gates (bun test, check, typecheck, format:check, build) all pass on the clean tree.

Example captured payloads

{
  "responses_request_q-1_epoch0": { "x-session-id": "wf-run-123.0.q-1", "x-openrouter-cache-salt": "wf-run-123:epoch-0" },
  "responses_request_q-1_epoch1": { "x-session-id": "wf-run-123.1.q-1", "x-openrouter-cache-salt": "wf-run-123:epoch-1" },
  "sanitized_sample": { "sample.id": "swe/atlas:task#7 v2", "x-session-id": "wf-run-123.0.swe-atlas-task-7-v2" },
  "long_sample": { "sample.id.length": 305, "x-session-id.length": 256, "suffix": "-4ca897508d02" },
  "retry_after_503": [
    { "status": 503, "x-session-id": "wf-run-123.0.q-1", "x-openrouter-cache-salt": "wf-run-123:epoch-0" },
    { "status": 200, "x-session-id": "wf-run-123.0.q-1", "x-openrouter-cache-salt": "wf-run-123:epoch-0:attempt-1" }
  ],
  "ori_exec_env": { "sampleId": "swe/atlas:task#7 v2", "ORI_OPENROUTER_SESSION_ID": "wf-run-123.1.swe-atlas-task-7-v2" },
  "no_session_run": { "x-session-id": null, "x-openrouter-cache-salt": "epoch-0" },
  "prefix_collision": { "wf-run-1234.0.q-1 matches parent wf-run-123": false }
}

📈 Live monitoring: Datadog, bench-harness Transaction attempt volume by @extra.session_id. Nested <run>.<epoch>.<sample> ids appear once the merged harness is deployed. Measured 2026-09-19 02:36 UTC: the active run still logs the flat run id rb-...-deep-swe. The PR is merged, so the description cannot be edited and the link lives here.

Full validation report: