Skip to content

feat: Mint metrics tokens for Datum - #240

Merged
prathameshkurunkar7 merged 4 commits into
developfrom
feat-datum-metrics-token
Aug 5, 2026
Merged

feat: Mint metrics tokens for Datum#240
prathameshkurunkar7 merged 4 commits into
developfrom
feat-datum-metrics-token

Conversation

@Aradhya-Tripathi

Copy link
Copy Markdown
Contributor

Datum stores metrics behind vmauth, which checks a Central-signed JWT before
anything reaches the store. It reads vm_access.metrics_extra_labels off the
token and hands them to VictoriaMetrics, which stamps them over whatever the
producer sent — so a pilot can't write as another resource even if it claims
one in the request body.

Central had no token for that. The existing mints are bench logins, site logins
and enrolment: all short-lived, none carrying a resource id.

This adds mint_metrics_token, and a metrics_token endpoint the pilot calls
with its usual X-Pilot-Token:

{
  "aud": "<pilot_credential_id>",
  "scope": "datum",
  "vm_access": {"metrics_extra_labels": ["resource_id=vm-abc123"]}
}

scope is the part that matters. Every downward token is signed with the same
key, so without it the gateway would treat a bench SID as permission to write
metrics. vmauth matches on scope: datum and rejects everything else.

The resource id comes from the credential's linked Asset. That's only bound
after Atlas provisions, so a pilot asking too early is told to wait rather than
handed a token whose samples couldn't be attributed to anything.

It's a separate endpoint rather than part of config because it expires — the
pilot re-fetches on a TTL, or when the gateway answers 401.

Verified the claim shape against a real vmauth: the token is accepted, and a
body claiming a different resource_id is overwritten with the one from the
token.

Aradhya-Tripathi and others added 4 commits August 5, 2026 14:42
Datum's metrics gateway is vmauth, which verifies a Central-signed JWT and
reads `vm_access.metrics_extra_labels` off it. VictoriaMetrics applies those
labels over whatever the producer sent, so a pilot cannot write as another
resource even if it claims one in the request body.

Central had no token for this. Its existing mints are bench logins, site
logins and enrolment, all short-lived and none carrying a resource id.

`scope` is what makes this safe to add: every downward token is signed with
the same key, so without one the gateway would accept a bench SID as a
licence to write metrics. It matches on `scope: datum` and rejects the rest.

The resource is bound after Atlas provisions, so minting before then is
refused rather than handing out a token whose samples could not be
attributed to anything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
They were the longest in both files -- nine lines where the neighbours run to
two or three. The facts that are not obvious from the code stay: what scope
guards against, and why minting is refused before Atlas binds the Asset.
No other mint has a test file of its own -- bench login and bootstrap are
covered inside test_enroll, site login inside test_atlas_sync. A dedicated
120-line file for this one was out of step, and larger than any test file in
the repo.

Two tests remain, beside the endpoint's siblings: the token carries the scope
and resource label, and minting is refused before Atlas binds the Asset. The
signature is not re-tested; test_sso already covers that every minted token
verifies against the JWKS.
A day meant every pilot re-fetching daily, and a pilot that cannot reach
Central for a day stops reporting metrics -- exactly when you want them.

The cost is that a leaked token is good for a year, and there is no
revocation list: pulling one back means rotating Central's signing key, which
invalidates every token it ever signed. Worth revisiting if pilots grow a way
to refresh unattended.
@prathameshkurunkar7
prathameshkurunkar7 merged commit 8ef4144 into develop Aug 5, 2026
2 of 3 checks passed
@prathameshkurunkar7
prathameshkurunkar7 deleted the feat-datum-metrics-token branch August 5, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants