Skip to content

ci(peer-http): calibrate shared-runner routing latency gates - #41

Merged
forhappy merged 1 commit into
mainfrom
codex/calibrate-routing-latency
Oct 2, 2026
Merged

forhappy merged 1 commit into
mainfrom
codex/calibrate-routing-latency

Conversation

@forhappy

@forhappy forhappy commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow up the merged #39 with the explicitly requested CI latency threshold adjustment. This PR changes only routing qualification and its documentation; production Rust is unchanged.

  • Blocking limits: median of all four runs p95 ≤150%, p99 ≤200% of baseline; throughput remains ≥90%.
  • Retain 110% latency review alerts in comparison.json, all raw samples, and record the blocking limits in both manifest and comparison.
  • Preserve both ownership modes, all four balanced pairs, every lane/sample, physical read/hop budgets, fresh authority checks, publication proofs and exact recovery.
  • Test both sides of the limits, alert reporting, missing repeats/modes and each partition's regression gates.

Calibration evidence and limits

The original-profile identical-binary control reported apparent 31.1% p95 /83.9% p99 increases with identical frozen source and executable hashes. The selected ceilings allow headroom over this observed shared-runner variation. Passing this coarse gate does not prove latency equivalence within 10% or a production SLO.

Historical results retain their original policy and failures. In particular, #39's old comparison failed forwarded commands at concurrency16 with p95/p99 ratios 1.140/1.244 leased and 1.213/1.465 object-only; throughput 0.950/0.987 passed. Those latency exceedances remain visible as alerts when replayed prospectively under this new policy. The raw audit confirms 484,384 latencies, 65,536 publication records, 98,304 exactly recovered commands.

Repeated identical-binary controls with CPU separation and independent providers also exceeded the old limits. They did not establish a provider/CPU fix or a code-versus-environment cause. No diagnostic instrumentation or experimental provider configuration is included here.

Verification

  • 14 qualification driver tests pass locally on the new main-based branch.
  • 1,082 local links/anchors and 104 Rust snippets pass; all three changed files are identical to the locally validated pre-merge commit.
  • Base: fbfd84f9c4dfcb6de497072efd9aafa5a6f409cc (merged perf(runtime): reduce forwarding admission and compaction publication latency #39). Candidate: a7db14750b24568a68efca98b5a743c87a8cc498.
  • Pre-merge comparison 36948754276 passes both modes and aggregate routing against pre-perf(runtime): reduce forwarding admission and compaction publication latency #39 main. Independent raw-data audit verifies all four pairs, 484,384 latency samples, 65,536 publication records and 98,304 exactly recovered commands, including read/hop budgets, coordination and source/binary provenance. The sole 110% review alert is object-only local-query/c16 p99 ratio 1.10058; p95 1.04769, throughput 0.98319. All other gated latency medians stay within 110%. Forwarded-command/c16 p99 is 294.49 → 281.48 ms leased and 278.71 → 269.86 ms object-only, with throughput ratios 1.022/1.035. These modest observed improvements are smaller than demonstrated CI variation and do not establish a stable gain. Candidate executable hashes exactly match the earlier controls; candidate tree is identical to this PR's tree. This is retained optimization evidence, not substituted for this PR's new baseline comparison.
  • PR #41 reference comparison passes leased, object-only, aggregate routing and Compose smoke against merged main. All eight PR checks pass, including Rust workspace/MSRV, contracts and website.
  • Independent raw-data audit verifies every original lane, all four pairs per mode, 484,384 latency samples, 65,536 publication records and 98,304 exactly recovered commands. Quantiles are recomputed from raw samples; read/hop budgets, publication sequences, nonoverlapping measurement windows, frozen source/binary hashes and recorded gate policy are verified.
  • One review alert remains: leased/local_query/c1 p99 ratio 1.13171 (+13.17%); p95 1.04142, throughput 0.99168. All other gated latency medians stay within 110%. No blocking gate failures; throughput remains above the original 90% limit in every applicable lane.
  • Frozen merge snapshot: a45fe7135169785ecef3cc28edb1e18754adc323; baseline: fbfd84f9c4dfcb6de497072efd9aafa5a6f409cc. Plans and full audit reports are retained outside the repository.

Plans and raw evidence remain outside the repository. PR #39 is left unchanged.

@forhappy
forhappy marked this pull request as ready for review October 2, 2026 02:13
@forhappy
forhappy merged commit 1914096 into main Oct 2, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant