Skip to content

Measure provider latency through response body consumption - #16

Merged
SamCT86 merged 2 commits into
mainfrom
promo/measure-full-provider-latency
Sep 22, 2026
Merged

SamCT86 merged 2 commits into
mainfrom
promo/measure-full-provider-latency

Conversation

@SamCT86

@SamCT86 SamCT86 commented Sep 22, 2026

Copy link
Copy Markdown
Owner

Problem

The OpenAI adapter captured latencyMs immediately after fetch() returned, before response.json() consumed the response body. That could under-report provider latency even though the runtime later uses the measured value as an acceptance bound.

Failure-first evidence

A deterministic regression advanced the injected clock by 75 ms while consuming the response body. On current main, the test failed with 0 !== 75.

Change

  • measure provider latency after the response body has been consumed
  • add a focused regression test for delayed body consumption
  • keep the existing provider timeout and cost accounting boundaries unchanged

Verification

Exact remote branch head was cloned and verified on STRATO:

  • npm test: 28/28 passed
  • npm run eval: 5/5 passed
  • git diff --check: passed
  • diff from current main: only src/runtime-gate.mjs plus the focused latency regression test

Boundary

This fixes provider-response latency accounting. It does not claim an end-to-end application SLO or distributed execution guarantee.

@SamCT86
SamCT86 merged commit ccc6abb into main Sep 22, 2026
1 check passed
@SamCT86
SamCT86 deleted the promo/measure-full-provider-latency branch September 22, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant