Repository navigation
Live speed: /live reports the current generation speed - #346
Merged
Merged
Conversation
/live used to give the speed of requests that finished in the last minute, so during a long request the menu bar had nothing to show. The gateway now keeps reading a successful stream after its first token and estimates the output as it arrives (text, reasoning text and tool arguments, with the same rule as the input estimate), multiplied by a per-upstream, per-model ratio. Each streaming request that finishes with reported usage calibrates that ratio (moving average, bounded 0.5..2, skipped when reasoning was hidden). Gemini's cumulative usage in the stream is used when it is higher. Each running request has a meter on the event bus (250 ms slots, 3 s window). /live gives every running request's tokens_per_sec and output_tokens (null before the first token and for non-streaming), and the total of all running requests: null when nothing runs, 0 when requests run but none is generating. The per-request speed recorded at the end is unchanged. Control protocol 49. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/livenow reports how fast running requests are generating right now, instead of the average of requests that finished in the last minute. During a long request the menu bar had nothing to show before.tw_gateway::pace).tw_observe::Meter, 250 ms slots, 3 s window).LiveView.tokens_per_sec= total of running requests (null when none runs, 0 when none is generating yet).RunningViewgainstokens_per_secandoutput_tokens(null before the first token / non-streaming).RequestFinished.tokens_per_sec) is unchanged.tw_engine::TextTokensis now public.Tests: meter unit tests, pace unit tests, bus
/livetests,tw-control/livetest updated, and a new gateway integration test (tests/live_speed.rs) that reads/livehalfway through a real stream and checks calibration after it ends.🤖 Generated with Claude Code