Export cost and governance data (on-demand snapshot; streaming as follow-up)

[tishachawla-jg]

Note: Tracked here on tokenops for visibility. The feature is implemented mostly in the control plane (that's where the data and routes live: theagentplane/control-plane), with a thin SDK/CLI helper in tokenops. Transferred from theagentplane/control-plane#17.

Summary

Let users get their data out of the control plane for FinOps / chargeback reporting and audit / compliance. The plane's SQLite holds it all (run-records, spend, governance events), but today there's no way to pull it into a spreadsheet, a cost tool, or an archive. Two modes; v1 is the on-demand snapshot.

What to export

• Run-records — cost per run, agent, and segment over a time window. This is the FinOps view (who spent what).
• Governance events — the audit trail: halts, MUTATE/INJECT decisions, which policy fired and why. This is the compliance view.
• Not the raw ledger accumulators (ledger_spent by budget/segment) — they're aggregate and low-value raw; the run-level breakdown above is what people actually want.

Modes

v1 — on-demand snapshot (pull)

• A GET /v1/export route on the control plane returning run-records and/or governance events, filtered by time range, run, agent, segment.
• Formats: CSV (finance/spreadsheets) and JSON / JSONL (pipelines).
• Surfaces: an Admin UI "Export" button and a thin SDK/CLI helper in tokenops on top of the route.

Follow-up — continuous / streaming (push)

• Push ledger/governance events to an external sink (webhook, S3, OpenTelemetry, a FinOps tool) as they happen. Bigger scope; separate issue once v1 lands.

Open decisions (comment before building)

• Which fields/columns matter most for the FinOps CSV (cost, tokens, cached, model, agent, segment, timestamps)?
• Auth/scope: export should require read scope and respect tenant ownership (ties to theagentplane/control-plane#6 and theagentplane/control-plane#16) — a caller must not export another tenant's spend.
• Pagination / size limits and streaming the response for large exports.

Acceptance (v1)

• Pull run-records + governance events over a time range in CSV and JSON.
• Scoped by read auth and tenant; no cross-tenant export.
• Admin "Export" button + SDK/CLI helper.
• Documented.

Related

theagentplane/control-plane#6 (ownership-scoped authorization — export must respect it), theagentplane/control-plane#16 (roles: who may export), #141 (cost attribution feeds what's exportable).

[RjyavardhanSingh]

on it

[RjyavardhanSingh]

• run_id, agent, status, cost_micros, steps, started_at, ended_at, duration_s, dims, halt_reason, detector, governance_events these are the fields had most weights out of all and recommended for CSV exports.

• duration will computed and for JSON/JSONL we send everything and user can have what ever they want?

• dims and governance_events are nested objects. For CSV, flatten dims into a JSON string column (e.g. {"tenant":"acme","project":"search"}), and governance_events into a JSON array string. Spreadsheets handle JSON strings fine, and pipelines can parse them.

@tishachawla-jg

[RjyavardhanSingh]

• For Auth we should built it on top of control-plane auth, it exacepts same headers as other modules
• We should trust it for control plane's auth layer and tenant ownership

[RjyavardhanSingh]

For pagination part it may not be required like for use case if month of runs 50k+ rows it can easily be embedded in json file and we should buffer memory with that much rows, it my take if it required esetially then i can implement?