diff --git a/CHANGELOG.md b/CHANGELOG.md index 84085e8..fc23aaf 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Added - `GET /api/v1/overview`, `/sessions`, `/costs` and `/models` accept `range`, the same five-key rolling window `/users` and `/tools` already took, and each echoes back the key it used. Defaults preserve today's behaviour instead of converging on one value: `/overview` and `/costs` default to `month` (the 30-day window they already applied), `/sessions` and `/models` to `all`, because they had no time filter and a `month` default would silently truncate every existing caller. Long ranges resolve against the `spans` ∪ `daily_usage` union at the raw-floor split, so `year` and `all` keep answering after retention has deleted the raw spans rather than repeating the `month` figure. On `/costs`, explicit `from`/`to` still beat the range key and the response then echoes `"range": null` ([ADR-0014](docs/decisions/0014-overview-single-range-selector.md)) - `GET /api/v1/sessions` returns `covered_since`. A session row needs a start time, model and status, none of which the roll-up keeps, so the list is raw-only; a range reaching past the raw floor is clamped and the field names the instant the list actually starts from (`null` when the range is fully covered). The Overview's Sessions block states that window in one line. The session *count* KPI is unaffected — `daily_usage` carries `session_id`, so counting distinct sessions across the union is exact +- `GET /api/v1/history` accepts `range` too — the last read endpoint that did not. It defaults to `month`, the window `from`/`to` already defaulted to, so a caller that passes no range sees exactly what it saw before; explicit `from`/`to` still win and the response then echoes `"range": null`, the precedence `/costs` sets. At `day`, `week` and `month` granularity `buckets` and `by_model` resolve over the `spans` ∪ `daily_usage` union at the raw-floor split, so the History chart no longer stops a month back while the Costs panel beside it charts a year. `hour` stays raw-only and reports it in `covered_since`: the roll-up consumes whole UTC days and cannot produce a sub-day bucket, and day-shaped data under an hour label would be worse than a stated shortfall. `from`/`to` in the response echo the resolved bounds and are `null` on a side the window does not bound ([ADR-0014](docs/decisions/0014-overview-single-range-selector.md)) +- `GET /api/v1/history` returns `heatmap_covered_since`. Both heatmaps resolve hour of day, which `daily_usage` does not keep, so they stay raw-only however far back the charts above them now reach. The History page states that window under each one instead of drawing empty cells that read as "no activity" for days that were merely rolled up ### Fixed - `GET /api/v1/overview`'s `users_count` obeyed neither the time window nor the `user_id` filter — it was a bare `SELECT COUNT(DISTINCT user_id) FROM spans`, so the "Users" KPI answered all-time and unscoped next to four KPIs that did not. It now counts the distinct principals active in the selected range, and counts unattributed spans as the single `__anonymous__` principal the users list shows rather than as zero @@ -20,6 +22,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ### Changed - The Overview is one window instead of five. A single range switcher in the header — its own `cotel_overview_range` cookie, so it does not move the Users or Tools page — scopes every figure on the page. Previously the KPIs showed 30 days, the Sessions and Models blocks showed all time, and only the KPI labels said which, as a literal `(30d)` baked into the string; a reader comparing the Sessions KPI against the Models table below it was comparing 30 days against all time. Labels now take their suffix from the selected range, and `All` renders none - Overview section order is Users, History, Costs, Tools, Models, Sessions. A new Users block leads with the top 5 principals by spend in the selected range, and Sessions moves to the bottom as the one block that cannot honour a long range. The Costs block drops its inner by-model table — the Models block below it is the same data at full width +- The Overview's History block stops borrowing the sessions list's `covered_since` and reads the coverage the history response reports for its own series. The note it printed on `Year` and `All` is gone on every range the union now covers, and remains only where the chart is genuinely short — the `Day` range, which charts hourly. `RANGE_DAYS` goes with it: the page no longer derives `from`/`to` from the range key at all, so its copy of the window lengths cannot drift from the server's - The Overview's user-search typeahead is gone, and the `UserSearch` component with it. Scoping is reached from a user's page ("View activity"); `?user_id=` now shows a chip in the header naming the user and clearing the scope on click, instead of a page that was silently filtered with nothing on it to say so - A deploy now fails when the container does not come up. The Deploy workflow ended at `docker compose up -d`, which returns once the container has *started*, not once it works — so the last thing it observed of a deploy was `Up Less than a second (health: starting)` and it went green on that, reporting a container whose `storage.Open` had died identically to one serving traffic. It now runs `scripts/wait-for-healthy.sh`, which blocks on the container's own `HEALTHCHECK` and fails the job on `unhealthy`, on an exit, on a crash loop (in under a second, rather than waiting out the timeout — only restarts seen *during* the wait indict a deploy, since `up -d` leaves an already-current container in place and one that crashed once and recovered carries a restart count for the rest of its life, including while it legitimately replays a WAL), on a service that defines no healthcheck at all, or on a 120 s timeout — dumping `docker compose ps`, the last health-probe output and the container logs so the reason is in the run log instead of on the runner. `workflow_dispatch` takes a `health_timeout` input for the one deploy that legitimately needs longer: a start following a hard kill replays the WAL. The CI smoke job runs the same script in place of its `curl`-until-ready loop, so a break in the gate surfaces on a PR rather than on a deploy. Measured against a 109 MB copy of production, healthy at 6 s from cold with a 3.9 MB WAL to replay (the open itself 2.8 s, including the v10 migration) and 6 s on a redeploy of the warm database — the 6 s is the probe cadence, not the database - The image `HEALTHCHECK` gains `--start-interval=5s`. `--interval=30s` also governed the probes during `start-period`, so a container that was ready in two seconds still reported `starting` for thirty, and the deploy gate above would have waited out all of it diff --git a/README.md b/README.md index 9070c70..09a7da4 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ One Docker container. OTLP ingest on `:4318`, interactive analytics dashboard on - **Overview dashboard** — one range switcher in the header (All / Year / Month / Week / Day, default 30 days) that every figure on the page obeys: the KPI cards for sessions, users, total cost and token counts, and the Users / History / Costs / Tools / Models / Sessions blocks below them. The Users block ranks your top 5 principals by spend in the selected range. Arriving with `?user_id=` scopes the whole page to one user, with a chip in the header to clear it - **Sessions** — live table of every Claude Code session with user, model, duration, cost, and status (OK / ERROR); search by user and click any user to filter the table to their sessions -- **History** — time-series and daily-activity heatmaps for sessions and token spend over time +- **History** — time-series and daily-activity heatmaps for sessions and token spend over time. The day, week and month series keep charting after retention has rolled raw spans into daily totals; the hourly series and both heatmaps need a per-span timestamp, so they cover raw days only and say from when - **Costs** — cumulative spend chart + breakdown table by model - **Tools** — call counts, average duration, and error rate per tool (`Bash`, `Read`, `Edit`, …), scoped to a range switcher (All / Year / Month / Week / Day, default 30 days), plus a per-command breakdown of `Bash` calls - **Models** — token and cost breakdown across all Claude model variants diff --git a/docs/operations/api-reference.md b/docs/operations/api-reference.md index b07bec9..995d076 100644 --- a/docs/operations/api-reference.md +++ b/docs/operations/api-reference.md @@ -36,11 +36,11 @@ so a client can render its controls from the response alone. ## `range` on the summary endpoints -`GET /overview`, `GET /sessions`, `GET /costs` and `GET /models` accept the same -`range` parameter, with the same five keys, the same rolling-window semantics and -the same fall-back-don't-`400` rule +`GET /overview`, `GET /sessions`, `GET /costs`, `GET /models` and `GET /history` +accept the same `range` parameter, with the same five keys, the same +rolling-window semantics and the same fall-back-don't-`400` rule ([ADR-0014](../decisions/0014-overview-single-range-selector.md)). Each echoes -back the `range` it used. All four also accept `user_id`, which composes with +back the `range` it used. All five also accept `user_id`, which composes with `range` rather than overriding it. **Defaults preserve each endpoint's previous behaviour rather than converging on @@ -50,6 +50,7 @@ one value:** |---|---|---| | `GET /overview` | `month` | The 30-day window it always applied | | `GET /costs` | `month` | The 30-day window `from`/`to` already defaulted to | +| `GET /history` | `month` | Same — the window a bare request already answered | | `GET /sessions` | `all` | Had no time filter; a `month` default would truncate existing callers | | `GET /models` | `all` | Same | @@ -61,12 +62,13 @@ instead of silently repeating the `month` figure. Two consequences are worth knowing: -- **`GET /costs`: explicit bounds beat the range key.** When a request carries - `from` and/or `to`, those win and `range` is ignored — the narrower, more - specific statement is the one the caller meant. The response then echoes - `"range": null`. `top_sessions[].first_seen` degrades to the aggregate's day at - midnight UTC for rolled-up sessions, since `daily_usage` keeps no intra-day - timestamp; it is a lower bound on the real start, never a later one. +- **`GET /costs` and `GET /history`: explicit bounds beat the range key.** When a + request carries `from` and/or `to`, those win and `range` is ignored — the + narrower, more specific statement is the one the caller meant. The response + then echoes `"range": null`. On `/costs`, `top_sessions[].first_seen` degrades + to the aggregate's day at midnight UTC for rolled-up sessions, since + `daily_usage` keeps no intra-day timestamp; it is a lower bound on the real + start, never a later one. - **`GET /sessions` clamps and says so.** A session row needs a start time, model and status, none of which the roll-up carries, so the list is computed from raw spans alone. A range reaching past the raw floor is clamped, and the response @@ -79,8 +81,56 @@ Two consequences are worth knowing: range, with all unattributed spans counting as the single `__anonymous__` principal the users list shows. -`GET /history` is not part of this contract; it takes `from`/`to` and reads raw -spans only. +## `GET /history` + +Activity over time, bucketed at the requested `granularity`. + +| Param | Values | Default | Meaning | +|---|---|---|---| +| `granularity` | `hour` \| `day` \| `week` \| `month` | `day` | Bucket width for `buckets` and `by_model` | +| `range` | `all` \| `year` \| `month` \| `week` \| `day` | `month` | Rolling window, as above | +| `from`, `to` | `YYYY-MM-DD` | — | Explicit bounds; when either is present they win and `range` echoes `null` | +| `user_id` | user id \| `__anonymous__` | — | Scopes every figure to one principal | + +```json +{ + "granularity": "day", + "from": "2026-07-21", + "to": null, + "range": "month", + "buckets": [ + { "bucket": "2026-08-19", "sessions": 12, "spans": 940, "cost_usd": 8.21, "input_tokens": 1201, "output_tokens": 4402 } + ], + "by_model": [ + { "bucket": "2026-08-19", "model": "claude-opus-5", "cost_usd": 7.90, "spans": 612 } + ], + "heatmap": [ + { "date": "2026-08-19", "hour": 14, "count": 61, "cost_usd": 0.94 } + ], + "covered_since": null, + "heatmap_covered_since": null +} +``` + +`from` and `to` echo the resolved bounds, and are `null` for a side the window +does not bound — `range=all` reports `"from": null`, and any range-scoped request +reports `"to": null` because the window runs to request time. + +**Which parts span the roll-up.** At `day`, `week` and `month` granularity, +`buckets` and `by_model` are answered from the `spans` ∪ `daily_usage` union at +the raw-floor split, so `year` and `all` keep charting after retention has +deleted the raw spans. Two fields report where that stops: + +- **`covered_since`** clamps `buckets` and `by_model`. It is always `null` at + `day`, `week` and `month`. At **`hour`** it names the raw floor when the window + reaches past it: `daily_usage` buckets whole UTC days and cannot produce a + sub-day bucket, so an hour series is raw-only rather than day-shaped data under + an hour label. +- **`heatmap_covered_since`** clamps `heatmap`, which resolves hour of day at + every granularity and is therefore always raw-only. + +Both are RFC3339, and `null` when the selected window is fully covered by raw +spans. ## `GET /tools` diff --git a/frontend/src/api/index.ts b/frontend/src/api/index.ts index d298522..6d2973b 100644 --- a/frontend/src/api/index.ts +++ b/frontend/src/api/index.ts @@ -373,19 +373,31 @@ export interface HeatmapCell { export interface HistoryResponse { granularity: string - from: string - to: string + from: string | null + to: string | null buckets: HistoryBucket[] by_model: HistoryModelRow[] heatmap: HeatmapCell[] + range: string | null + covered_since: string | null + heatmap_covered_since: string | null } -export function useHistory(granularity: string, from?: string, to?: string, userId?: string) { +export function useHistory( + granularity: string, + from?: string, + to?: string, + userId?: string, + range?: string, +) { const params = new URLSearchParams({ granularity }) if (from) params.set('from', from) if (to) params.set('to', to) if (userId) params.set('user_id', userId) - return useSWR(`/api/v1/history?${params.toString()}`, fetcher) + if (range) params.set('range', range) + return useSWR(`/api/v1/history?${params.toString()}`, fetcher, { + keepPreviousData: true, + }) } export interface TokenItem { diff --git a/frontend/src/pages/History.module.css b/frontend/src/pages/History.module.css index 0c4437c..680d7f7 100644 --- a/frontend/src/pages/History.module.css +++ b/frontend/src/pages/History.module.css @@ -135,6 +135,12 @@ margin: 0 0 var(--space-2); } +.coverageNote { + font-size: var(--text-sm); + color: var(--color-text-3); + margin: var(--space-3) 0 0; +} + /* ---- Calendar heatmap ---- */ .calWrap { position: relative; diff --git a/frontend/src/pages/History.tsx b/frontend/src/pages/History.tsx index 674e708..127fc21 100644 --- a/frontend/src/pages/History.tsx +++ b/frontend/src/pages/History.tsx @@ -311,6 +311,19 @@ function tickLabel(gran: Granularity, bucket: string): string { return bucket.slice(5) } +// The daily roll-up keeps no intra-day timestamp, so anything that resolves an +// hour — the hourly series, and both heatmaps at any granularity — stays +// raw-only however far back the day, week and month series reach. +function CoverageNote({ since, subject }: { since: string | null; subject: string }) { + if (!since) return null + return ( +

+ Plotted from {new Date(since).toLocaleDateString()} — {subject}, and earlier days in this + window survive only as whole-day totals. +

+ ) +} + export default function History() { const [gran, setGran] = useState('day') const [from, setFrom] = useState(daysAgo(30)) @@ -468,6 +481,7 @@ export default function History() { )} + {/* Calendar heatmap */} @@ -475,7 +489,10 @@ export default function History() { {calDays.length === 0 ? ( ) : ( - + <> + + + )} @@ -484,7 +501,10 @@ export default function History() { {data.heatmap.length === 0 ? ( ) : ( - + <> + + + )} diff --git a/frontend/src/pages/Overview.tsx b/frontend/src/pages/Overview.tsx index 07be9e3..65a5610 100644 --- a/frontend/src/pages/Overview.tsx +++ b/frontend/src/pages/Overview.tsx @@ -20,18 +20,6 @@ import styles from './Overview.module.css' const RANGE_COOKIE = 'cotel_overview_range' const ANON_ID = '__anonymous__' -// /history is bounded by from/to rather than the range key, so "All" has to name -// a start date. Nothing cotel can store predates this by decades. -const HISTORY_EPOCH = '2000-01-01' - -const RANGE_DAYS: Record = { - all: null, - year: 365, - month: 30, - week: 7, - day: 1, -} - interface SectionProps { range: RangeKey userId?: string @@ -43,10 +31,6 @@ function fmtTokens(n: number): string { return String(n) } -function isoDate(d: Date): string { - return d.toISOString().slice(0, 10) -} - function formatDay(iso: string): string { return new Date(iso).toLocaleDateString() } @@ -92,14 +76,14 @@ function UsersSection({ range }: { range: RangeKey }) { ) } -function HistorySection({ range, userId, coveredSince }: SectionProps & { coveredSince: string | null }) { - const days = RANGE_DAYS[range] - const from = useMemo( - () => (days === null ? HISTORY_EPOCH : isoDate(new Date(Date.now() - days * 86400_000))), - [days], +function HistorySection({ range, userId }: SectionProps) { + const { data, isLoading, error } = useHistory( + range === 'day' ? 'hour' : 'day', + undefined, + undefined, + userId, + range, ) - const to = useMemo(() => isoDate(new Date()), []) - const { data, isLoading, error } = useHistory(range === 'day' ? 'hour' : 'day', from, to, userId) if (isLoading && !data) return if (error) return @@ -108,10 +92,10 @@ function HistorySection({ range, userId, coveredSince }: SectionProps & { covere return ( <> - {coveredSince && ( + {data.covered_since && (

- Charted from {formatDay(coveredSince)} — this chart is built from raw spans, and earlier - days in this range survive only as daily totals. + Charted from {formatDay(data.covered_since)} — hourly buckets are built from raw spans, and + earlier days in this range survive only as whole-day totals.

)} @@ -336,12 +320,6 @@ export default function Overview() { const { data, error, isLoading, isValidating, mutate } = useOverview(paused ? 0 : 30_000, userId, range) - // History is charted from raw spans, so it starts at the same raw floor the - // sessions list reports. Same SWR key as the Sessions section below, so the - // two share one request rather than issuing two. - const { data: sessions } = useSessions(1, 5, 'start_time', 'desc', userId, range) - const coveredSince = sessions?.covered_since ?? null - const userParam = userId ? `?user_id=${encodeURIComponent(userId)}` : '' const suffix = RANGE_SUFFIX[range] const scoped = (label: string) => (suffix ? `${label} (${suffix})` : label) @@ -403,7 +381,7 @@ export default function Overview() { )} - + diff --git a/internal/api/handler.go b/internal/api/handler.go index 9ca4da2..454e72e 100644 --- a/internal/api/handler.go +++ b/internal/api/handler.go @@ -1298,13 +1298,26 @@ type heatmapCell struct { type historyResponse struct { Granularity string `json:"granularity"` - From string `json:"from"` - To string `json:"to"` + From *string `json:"from"` + To *string `json:"to"` Buckets []historyBucket `json:"buckets"` ByModel []historyModelRow `json:"by_model"` Heatmap []heatmapCell `json:"heatmap"` + // Range is the range key that scoped this response, or null when explicit + // from/to bounds superseded it (ADR-0014). + Range *string `json:"range"` + // CoveredSince names the start of the window buckets and by_model actually + // answer for, or null when the selected window is fully covered. Only "hour" + // can fall short: the roll-up consumes whole UTC days, so sub-day buckets + // exist for raw spans alone. Coarser granularities span the union and always + // report null. + CoveredSince *string `json:"covered_since"` + // HeatmapCoveredSince is the same clamp for heatmap, which resolves hour of + // day at every granularity and so is raw-only whatever the caller asked for. + HeatmapCoveredSince *string `json:"heatmap_covered_since"` } +// historyBucketExpr buckets raw spans by their start time. func historyBucketExpr(gran string) string { switch gran { case "hour": @@ -1318,6 +1331,42 @@ func historyBucketExpr(gran string) string { } } +// historyUnionBucketExpr buckets the usageCTE's whole-day rows. There is no +// "hour" case by construction: daily_usage keeps no intra-day timestamp, so an +// hour bucket cannot be built from the aggregate side at all. +func historyUnionBucketExpr(gran string) string { + switch gran { + case "week": + return "strftime(date_trunc('week', CAST(day AS TIMESTAMP))::TIMESTAMP, '%Y-%m-%d')" + case "month": + return "strftime(day, '%Y-%m')" + default: + return "strftime(day, '%Y-%m-%d')" + } +} + +// rawWindow builds the raw-spans time clause for from/to, either of which may be +// nil for an unbounded side. +func rawWindow(from, to *time.Time) (clause string, args []any) { + if from != nil { + clause += " AND start_time >= ?" + args = append(args, *from) + } + if to != nil { + clause += " AND start_time <= ?" + args = append(args, *to) + } + return clause, args +} + +func isoDay(t *time.Time) *string { + if t == nil { + return nil + } + s := t.Format("2006-01-02") + return &s +} + func (h *Handler) handleHistory(w http.ResponseWriter, r *http.Request) { gran := r.URL.Query().Get("granularity") switch gran { @@ -1326,18 +1375,70 @@ func (h *Handler) handleHistory(w http.ResponseWriter, r *http.Request) { gran = "day" } - from, to := parseDateRange(r) - bucketExpr := historyBucketExpr(gran) + // "month" is the default because it is the window parseDateRange already + // applied when a caller passed neither from nor to. + rangeKey, since := parseRangeDefault(r, "month") + from, to, explicit := explicitDateRange(r) + echo := &rangeKey + if explicit { + echo = nil + } else { + from, to = since, nil + } + + resp := historyResponse{ + Granularity: gran, + From: isoDay(from), + To: isoDay(to), + Buckets: []historyBucket{}, + ByModel: []historyModelRow{}, + Heatmap: []heatmapCell{}, + Range: echo, + HeatmapCoveredSince: h.rawCoveredSince(r, from), + } - var resp historyResponse - resp.Granularity = gran - resp.From = from.Format("2006-01-02") - resp.To = to.Format("2006-01-02") + if gran == "hour" { + resp.CoveredSince = resp.HeatmapCoveredSince + h.historyRawSeries(&resp, r, gran, from, to) + } else { + h.historyUnionSeries(&resp, r, gran, from, to) + } uidClause, uid := userIDClause(r) - baseArgs := []any{from, to} + window, args := rawWindow(from, to) if uid != "" { - baseArgs = append(baseArgs, uid) + args = append(args, uid) + } + hrows, _ := h.db.Query(` + SELECT + strftime(CAST(start_time AS TIMESTAMP), '%Y-%m-%d') AS date, + CAST(strftime(CAST(start_time AS TIMESTAMP), '%H') AS INTEGER) AS hour, + COUNT(*) AS count, + COALESCE(SUM(cost_usd), 0) AS cost_usd + FROM spans + WHERE TRUE`+window+uidClause+` + GROUP BY date, hour ORDER BY date ASC, hour ASC + `, args...) + if hrows != nil { + defer hrows.Close() + for hrows.Next() { + var c heatmapCell + _ = hrows.Scan(&c.Date, &c.Hour, &c.Count, &c.CostUSD) + resp.Heatmap = append(resp.Heatmap, c) + } + } + + jsonOK(w, resp) +} + +// historyRawSeries fills buckets and by_model from raw spans, for the "hour" +// granularity the roll-up cannot express. +func (h *Handler) historyRawSeries(resp *historyResponse, r *http.Request, gran string, from, to *time.Time) { + bucketExpr := historyBucketExpr(gran) + uidClause, uid := userIDClause(r) + window, args := rawWindow(from, to) + if uid != "" { + args = append(args, uid) } brows, _ := h.db.Query(fmt.Sprintf(` @@ -1349,10 +1450,9 @@ func (h *Handler) handleHistory(w http.ResponseWriter, r *http.Request) { COALESCE(SUM(input_tokens), 0) AS input_tokens, COALESCE(SUM(output_tokens), 0) AS output_tokens FROM spans - WHERE start_time >= ? AND start_time <= ?%s + WHERE TRUE%s%s GROUP BY bucket ORDER BY bucket ASC - `, bucketExpr, uidClause), baseArgs...) - resp.Buckets = []historyBucket{} + `, bucketExpr, window, uidClause), args...) if brows != nil { defer brows.Close() for brows.Next() { @@ -1369,10 +1469,9 @@ func (h *Handler) handleHistory(w http.ResponseWriter, r *http.Request) { COALESCE(SUM(cost_usd), 0) AS cost_usd, COUNT(*) AS spans FROM spans - WHERE start_time >= ? AND start_time <= ? AND model IS NOT NULL AND model <> ''%s - GROUP BY bucket, model ORDER BY bucket ASC, SUM(cost_usd) DESC - `, bucketExpr, uidClause), baseArgs...) - resp.ByModel = []historyModelRow{} + WHERE model IS NOT NULL AND model <> ''%s%s + GROUP BY bucket, model ORDER BY bucket ASC, SUM(cost_usd) DESC, model ASC + `, bucketExpr, window, uidClause), args...) if mrows != nil { defer mrows.Close() for mrows.Next() { @@ -1381,26 +1480,52 @@ func (h *Handler) handleHistory(w http.ResponseWriter, r *http.Request) { resp.ByModel = append(resp.ByModel, m) } } +} - hrows, _ := h.db.Query(` - SELECT - strftime(CAST(start_time AS TIMESTAMP), '%Y-%m-%d') AS date, - CAST(strftime(CAST(start_time AS TIMESTAMP), '%H') AS INTEGER) AS hour, - COUNT(*) AS count, - COALESCE(SUM(cost_usd), 0) AS cost_usd - FROM spans - WHERE start_time >= ? AND start_time <= ?`+uidClause+` - GROUP BY date, hour ORDER BY date ASC, hour ASC - `, baseArgs...) - resp.Heatmap = []heatmapCell{} - if hrows != nil { - defer hrows.Close() - for hrows.Next() { - var c heatmapCell - _ = hrows.Scan(&c.Date, &c.Hour, &c.Count, &c.CostUSD) - resp.Heatmap = append(resp.Heatmap, c) +// historyUnionSeries fills buckets and by_model from the spans ∪ daily_usage +// union, so a window reaching past the raw floor keeps charting instead of +// stopping there. +func (h *Handler) historyUnionSeries(resp *historyResponse, r *http.Request, gran string, from, to *time.Time) { + bucketExpr := historyUnionBucketExpr(gran) + f := newUsageFilter(r, from, to) + cte := f.cte() + + brows, _ := h.db.Query(cte+fmt.Sprintf(` +SELECT + %s AS bucket, + COUNT(DISTINCT session_id) AS sessions, + CAST(SUM(spans) AS BIGINT) AS spans, + COALESCE(SUM(cost), 0) AS cost_usd, + COALESCE(SUM(input_tokens), 0) AS input_tokens, + COALESCE(SUM(output_tokens), 0) AS output_tokens +FROM usage +GROUP BY bucket ORDER BY bucket ASC +`, bucketExpr), f.args...) + if brows != nil { + defer brows.Close() + for brows.Next() { + var b historyBucket + _ = brows.Scan(&b.Bucket, &b.Sessions, &b.Spans, &b.CostUSD, &b.InputTokens, &b.OutputTokens) + resp.Buckets = append(resp.Buckets, b) } } - jsonOK(w, resp) + mrows, _ := h.db.Query(cte+fmt.Sprintf(` +SELECT + %s AS bucket, + model, + COALESCE(SUM(cost), 0) AS cost_usd, + CAST(SUM(spans) AS BIGINT) AS spans +FROM usage +WHERE model IS NOT NULL +GROUP BY bucket, model ORDER BY bucket ASC, SUM(cost) DESC, model ASC +`, bucketExpr), f.args...) + if mrows != nil { + defer mrows.Close() + for mrows.Next() { + var m historyModelRow + _ = mrows.Scan(&m.Bucket, &m.Model, &m.CostUSD, &m.Spans) + resp.ByModel = append(resp.ByModel, m) + } + } } diff --git a/internal/api/history_range_test.go b/internal/api/history_range_test.go new file mode 100644 index 0000000..32baa1c --- /dev/null +++ b/internal/api/history_range_test.go @@ -0,0 +1,325 @@ +package api_test + +import ( + "net/http" + "testing" + "time" + + "github.com/Flopsstuff/cotel/internal/api" + "github.com/Flopsstuff/cotel/internal/storage" +) + +// sumBuckets totals one numeric field across the bucket series. +func sumBuckets(t *testing.T, body map[string]any, field string) float64 { + t.Helper() + var total float64 + for _, it := range bucketList(t, body) { + v, ok := it.(map[string]any)[field].(float64) + if !ok { + t.Fatalf("buckets[].%s: want number, got %v", field, it) + } + total += v + } + return total +} + +// sumByModel totals by_model spans per model. by_model is one row per (bucket, +// model), so a model that spans several buckets has several rows. +func sumByModel(t *testing.T, body map[string]any) map[string]float64 { + t.Helper() + rows, _ := body["by_model"].([]any) + out := map[string]float64{} + for _, it := range rows { + m := it.(map[string]any) + out[m["model"].(string)] += m["spans"].(float64) + } + return out +} + +func bucketList(t *testing.T, body map[string]any) []any { + t.Helper() + items, ok := body["buckets"].([]any) + if !ok { + t.Fatalf("buckets: want a list, got %v", body["buckets"]) + } + return items +} + +// TestHistory_RangeDefaultsToMonth pins the default that preserves the previous +// behaviour: parseDateRange already answered a bare /history over the last 30 +// days, so a caller that passes no range must still see exactly that. +func TestHistory_RangeDefaultsToMonth(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + for _, path := range []string{"/api/v1/history", "/api/v1/history?range=fortnight", "/api/v1/history?range="} { + code, body := getJSON(t, h, path) + if code != http.StatusOK { + t.Fatalf("%s: want 200, got %d: %v", path, code, body) + } + if body["range"] != "month" { + t.Errorf("%s: range echo: want month, got %v", path, body["range"]) + } + if got := len(bucketList(t, body)); got != 2 { + t.Errorf("%s: want the 2 raw days, got %d buckets", path, got) + } + if got := sumBuckets(t, body, "cost_usd"); got != 3 { + t.Errorf("%s: cost over buckets: want 3, got %v", path, got) + } + } +} + +// TestHistory_DayGranularityAcrossUnion is the criterion this ticket exists for: +// on a long range the day series keeps charting past the raw floor instead of +// stopping there, and the floor day is counted exactly once. +func TestHistory_DayGranularityAcrossUnion(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + cases := []struct { + rangeKey string + buckets int + spans float64 + cost float64 + sessions float64 + }{ + {"day", 1, 1, 1, 1}, + {"week", 2, 2, 3, 2}, + {"month", 2, 2, 3, 2}, + {"year", 3, 5, 7, 3}, + {"all", 4, 10, 15, 4}, + } + for _, tc := range cases { + t.Run(tc.rangeKey, func(t *testing.T) { + code, body := getJSON(t, h, "/api/v1/history?granularity=day&range="+tc.rangeKey) + if code != http.StatusOK { + t.Fatalf("want 200, got %d: %v", code, body) + } + if body["range"] != tc.rangeKey { + t.Errorf("range echo: want %q, got %v", tc.rangeKey, body["range"]) + } + if body["granularity"] != "day" { + t.Errorf("granularity echo: want day, got %v", body["granularity"]) + } + if got := len(bucketList(t, body)); got != tc.buckets { + t.Errorf("buckets: want %d, got %d (%v)", tc.buckets, got, body["buckets"]) + } + if got := sumBuckets(t, body, "spans"); got != tc.spans { + t.Errorf("spans: want %v, got %v", tc.spans, got) + } + if got := sumBuckets(t, body, "cost_usd"); got != tc.cost { + t.Errorf("cost_usd: want %v, got %v", tc.cost, got) + } + if got := sumBuckets(t, body, "sessions"); got != tc.sessions { + t.Errorf("sessions: want %v, got %v", tc.sessions, got) + } + // The union covers a day-granularity window whole, so there is no + // shortfall left to report. + if body["covered_since"] != nil { + t.Errorf("covered_since: want null at day granularity, got %v", body["covered_since"]) + } + }) + } +} + +// TestHistory_FloorDayNotDoubleCounted pins the boundary directly: the raw +// floor's calendar day also carries a $99 aggregate row, which stands for the +// same day the raw span already covers. +func TestHistory_FloorDayNotDoubleCounted(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + _, body := getJSON(t, h, "/api/v1/history?granularity=day&range=all") + byDay := pairs(t, body, "buckets", "bucket", "cost_usd") + floorDay := floorNoon().UTC().Format("2006-01-02") + if got := byDay[floorDay]; got != 2 { + t.Errorf("floor day %s: want cost 2 (raw only), got %v — full map %v", floorDay, got, byDay) + } + spansByDay := pairs(t, body, "buckets", "bucket", "spans") + if got := spansByDay[floorDay]; got != 1 { + t.Errorf("floor day %s: want 1 span (raw only), got %v", floorDay, got) + } +} + +// TestHistory_HourStaysRawOnly holds the line daily_usage cannot cross: it +// buckets whole UTC days, so an hour series can only be raw. The response must +// clamp and say where its coverage starts rather than relabel day-shaped data. +func TestHistory_HourStaysRawOnly(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + _, body := getJSON(t, h, "/api/v1/history?granularity=hour&range=all") + if body["granularity"] != "hour" { + t.Fatalf("granularity echo: want hour, got %v", body["granularity"]) + } + if got := len(bucketList(t, body)); got != 2 { + t.Errorf("buckets: want the 2 raw hours, got %d (%v)", got, body["buckets"]) + } + if got := sumBuckets(t, body, "spans"); got != 2 { + t.Errorf("spans: want 2 raw spans, got %v — aggregate rows leaked into an hour bucket", got) + } + if got := sumBuckets(t, body, "cost_usd"); got != 3 { + t.Errorf("cost_usd: want 3, got %v", got) + } + for _, it := range bucketList(t, body) { + if b := it.(map[string]any)["bucket"].(string); len(b) != len("2006-01-02 15:00") { + t.Errorf("bucket %q is not hour-shaped", b) + } + } + + models := sumByModel(t, body) + if len(models) != 1 || models["sonnet"] != 2 { + t.Errorf("by_model: want only the raw sonnet spans, got %v", models) + } + + covered, ok := body["covered_since"].(string) + if !ok { + t.Fatalf("covered_since: want the raw floor, got %v", body["covered_since"]) + } + ts, err := time.Parse(time.RFC3339, covered) + if err != nil { + t.Fatalf("covered_since %q is not RFC3339: %v", covered, err) + } + if d := ts.Sub(floorNoon()); d < -time.Second || d > time.Second { + t.Errorf("covered_since: want the raw floor %s, got %s", floorNoon(), ts) + } +} + +// TestHistory_HourReportsFullCoverageWhenRangeIsShort keeps the note honest in +// the other direction: a window that never reaches past the raw floor has +// nothing to disclose. +func TestHistory_HourReportsFullCoverageWhenRangeIsShort(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + cases := map[string]bool{"day": false, "week": false, "month": false, "year": true, "all": true} + for rangeKey, wantClamp := range cases { + t.Run(rangeKey, func(t *testing.T) { + _, body := getJSON(t, h, "/api/v1/history?granularity=hour&range="+rangeKey) + if got := body["covered_since"] != nil; got != wantClamp { + t.Errorf("covered_since set = %v, want %v (got %v)", got, wantClamp, body["covered_since"]) + } + }) + } +} + +// TestHistory_HeatmapIsAlwaysRawOnly covers the one series that stays raw at +// every granularity: it resolves hour of day, which the roll-up does not keep. +func TestHistory_HeatmapIsAlwaysRawOnly(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + _, body := getJSON(t, h, "/api/v1/history?granularity=day&range=all") + cells, _ := body["heatmap"].([]any) + if len(cells) != 2 { + t.Errorf("heatmap: want the 2 raw spans' cells, got %d (%v)", len(cells), cells) + } + if body["heatmap_covered_since"] == nil { + t.Error("heatmap_covered_since: want the raw floor on range=all, got null") + } + + _, month := getJSON(t, h, "/api/v1/history?granularity=day&range=month") + if month["heatmap_covered_since"] != nil { + t.Errorf("heatmap_covered_since: want null on a fully-raw window, got %v", month["heatmap_covered_since"]) + } +} + +// TestHistory_ExplicitBoundsBeatRange matches the precedence /costs sets: the +// narrower statement wins and the response echoes a null range. +func TestHistory_ExplicitBoundsBeatRange(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + from := floorNoon().UTC().Format("2006-01-02") + _, body := getJSON(t, h, "/api/v1/history?granularity=day&range=all&from="+from) + if body["range"] != nil { + t.Errorf("range echo: want null when from/to win, got %v", body["range"]) + } + if body["from"] != from { + t.Errorf("from echo: want %q, got %v", from, body["from"]) + } + if got := len(bucketList(t, body)); got != 2 { + t.Errorf("buckets: want only the 2 days inside from/to, got %d (%v)", got, body["buckets"]) + } + if got := sumBuckets(t, body, "cost_usd"); got != 3 { + t.Errorf("cost_usd: want 3, got %v — range=all leaked past the explicit bound", got) + } +} + +// TestHistory_CoarseGranularitiesSpanTheUnion checks week and month roll the +// union's whole-day rows up rather than falling back to raw spans, and that the +// roll-up's 'unknown' sentinel never surfaces as a model. +func TestHistory_CoarseGranularitiesSpanTheUnion(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + addUsageRow(t, db, usageRow{ + day: floorNoon().AddDate(0, 0, -55), session: storage.UnknownSentinel, + model: storage.UnknownSentinel, tool: storage.UnknownSentinel, + spans: 11, cost: 1, + }) + h := api.New(ro) + + for _, gran := range []string{"week", "month"} { + t.Run(gran, func(t *testing.T) { + _, body := getJSON(t, h, "/api/v1/history?range=all&granularity="+gran) + if got := sumBuckets(t, body, "spans"); got != 21 { + t.Errorf("spans: want 21 across the union, got %v", got) + } + if got := sumBuckets(t, body, "cost_usd"); got != 16 { + t.Errorf("cost_usd: want 16 across the union, got %v", got) + } + if body["covered_since"] != nil { + t.Errorf("covered_since: want null, got %v", body["covered_since"]) + } + + models := sumByModel(t, body) + if _, ok := models[storage.UnknownSentinel]; ok { + t.Errorf("by_model leaked the roll-up sentinel: %v", models) + } + var total float64 + for _, n := range models { + total += n + } + if total != 10 { + t.Errorf("by_model spans: want 10 attributed spans, got %v (%v)", total, models) + } + }) + } +} + +// TestHistory_RangeComposesWithUserID checks the two filters intersect on both +// sides of the union rather than one overriding the other. +func TestHistory_RangeComposesWithUserID(t *testing.T) { + db, ro := openTestDB(t) + seedRangeFixture(t, db) + h := api.New(ro) + + cases := []struct { + path string + spans float64 + cost float64 + }{ + {"/api/v1/history?granularity=day&range=all&user_id=alice", 2, 3}, + {"/api/v1/history?granularity=day&range=all&user_id=bob", 3, 4}, + {"/api/v1/history?granularity=day&range=all&user_id=carol", 5, 8}, + {"/api/v1/history?granularity=day&range=month&user_id=bob", 0, 0}, + } + for _, tc := range cases { + t.Run(tc.path, func(t *testing.T) { + _, body := getJSON(t, h, tc.path) + if got := sumBuckets(t, body, "spans"); got != tc.spans { + t.Errorf("spans: want %v, got %v", tc.spans, got) + } + if got := sumBuckets(t, body, "cost_usd"); got != tc.cost { + t.Errorf("cost_usd: want %v, got %v", tc.cost, got) + } + }) + } +} diff --git a/internal/dashboard/static/index.html b/internal/dashboard/static/index.html index 2661485..ff05d37 100644 --- a/internal/dashboard/static/index.html +++ b/internal/dashboard/static/index.html @@ -5,8 +5,8 @@ cotel - - + +