frontend: attribute offered load to the selected worker namespace

[Pernekhan]

Problem

With namespace grouping, one frontend serves several worker namespaces (<stem>--<group>) for a single model. The frontend labels all of its own metrics with its own dynamo_namespace, which is the model stem — not the group a request landed on.

So every group's planner reads the same model-wide dynamo_frontend_requests_started_total and sees the whole model's demand rather than its own. Each planner then scales for traffic another group is actually serving.

Change

Frontend. New counter:

dynamo_frontend_worker_namespace_requests_started_total{model, worker_namespace}

incremented where the WorkerSet is chosen. That point sits before any queue or scheduler work, so it measures offered load — a request later rejected with 529 still counts as demand, which is what a scaling decision needs.

select_worker_set_with is split so the selection proper hands back the chosen WorkerSet alongside the extracted engine. That gives exactly one increment site covering every entry path, including the single-WorkerSet fast path, rather than one per typed wrapper.

Ungrouped models emit the same series shape with their single namespace, so old and new deployments look alike to the planner.

Planner. get_avg_request_count prefers the new counter filtered on its own namespace, and falls back to the model-wide requests_started_total when the metric has no series — a planner pointed at an older frontend is unaffected. Two existing tests pinned the query order and are updated for the extra leading query.

The histogram namespace test becomes a stem match instead of equality. TTFT/ITL series only exist model-wide; under equality a grouped planner reads zero for both and its SLA arms go dead.

The match is deliberately narrower than a bare prefix:

planner namespace	series namespace	matches
<stem>--g1	<stem>	yes — model-wide series
<stem>--g1	<stem>--g1	yes
<stem>--g1	<stem>--g2	no — sibling group
<stem>--g1	"" / absent	no
<stem>	<stem>	yes — unchanged from equality
aa--Foo-extra	aa--Foo	no — not a -- boundary

Rejecting an empty label matters in practice: on our cluster deepseek-ai/DeepSeek-V3.2-dynamo emits six frontend series carrying no dynamo_namespace at all. Treating absent as a wildcard would sum those into every planner's reading for that model. Equality rejected them before, so this preserves the prior behaviour rather than loosening it.

Why not the router's counter

RouterRequestMetrics::from_component is a process-global OnceLock (kv_router/metrics.rs:836-880), so metrics_source="router" cannot carry per-group series. It also increments only after admission (request_guard.rs:309), which measures accepted load, not offered load.

Cardinality

Two labels, one series per (model, group). Groups are expected to stay in the low single digits per model.

Validation

• cargo check -p dynamo-llm clean, no new warnings.
• cargo test -p dynamo-llm --lib discovery::model — 111 passed. New test asserts the fast path counts, that the weighted path credits the group actually drawn, and that a non-selection (extract returns None) does not count.
• test_prometheus.py run against a matched module build, before vs after this change: identical failure set (11 pre-existing, from running a v1.4.1 test file against a July-9 planner image), +10 passing — the truth table above plus both branches of the new offered-load query.

🤖 Generated with Claude Code

https://claude.ai/code/session_017rS1uFFKT2xV4x63Amfvtw

[Pernekhan]

Measured on a real two-group family, not just unit-tested

A two-group dynamo family was stood up on our cluster to exercise this directly: Pernekhan/DeepSeek-V4-Flash, 1×B300 agg workers, one prefix-routed frontend, and a per-group planner. I verified each of the following independently.

The setup is exactly the case this PR addresses:

component	DYN_NAMESPACE
worker A	Pernekhan--DeepSeek-V4-Flash--g1
worker B	Pernekhan--DeepSeek-V4-Flash--g2
planner	Pernekhan--DeepSeek-V4-Flash--g1
frontend	pod label nvidia.com/dynamo-namespace: Pernekhan--DeepSeek-V4-Flash — the stem

A prefix frontend is model-wide, so it can only carry the stem. Confirmed in the metric store:

count by (dynamo_namespace, model) (dynamo_frontend_requests_started_total{model=~".*Pernekhan.*"})
=> {dynamo_namespace="Pernekhan--DeepSeek-V4-Flash", model="Pernekhan/DeepSeek-V4-Flash"}  1

One series, labelled with the stem. The planner filters on ...--g1. Equality never matches.

The planner is consequently blind on every arm, from its own logs:

WARN traffic_metrics.get_avg_request_count: No prometheus metric data available for
     dynamo_frontend_requests_started_total with model Pernekhan/DeepSeek-V4-Flash
WARN traffic_metrics._get_average_metric: No prometheus metric data available for
     dynamo_frontend_time_to_first_token_seconds ...
WARN traffic_metrics._get_average_metric: No prometheus metric data available for
     dynamo_frontend_inter_token_latency_seconds ...
WARN base._collect_traffic: Metrics gap detected around the request counter
     (raw num_req=0.0); skipping throughput tick
INFO base._log_decision_summary: [summary] HOLD | current: prefill=0 decode=1 | load_reason=no_fpm_data

TTFT and ITL fall to use 0 instead, the request counter falls back to completed, and the throughput tick is skipped outright. The planner holds because it can see nothing — not because nothing is happening.

Demand is genuinely per-group and genuinely observable — just not on the arm the planner reads:

sum by (dynamo_namespace) (dynamo_component_requests_total{dynamo_namespace=~"Pernekhan.*"})
=> Pernekhan--DeepSeek-V4-Flash--g1  16
   Pernekhan--DeepSeek-V4-Flash--g2  10

Traffic through the single prefix frontend splits across both groups (consistent with weighted-random WorkerSet selection at equal worker counts — the pick this PR attributes). So the information exists; there is simply no frontend-side counter carrying it. That is what worker_namespace_requests_started_total adds, and the segment-prefix match is what lets the model-wide TTFT/ITL histograms land.

One caveat worth recording for anyone verifying on an agg-only family: the load-based arm is dead there for an unrelated reason — No FPM data for either prefill or decode, skipping, with prefill=0 decode=1. An agg family has no prefill worker, so the disagg load path has nothing to read and every tick logs HOLD | load_reason=no_fpm_data. The throughput arm is the only one that can work on such a family, which makes it the arm to watch — and it is precisely the one this PR restores.