diff --git a/CHANGELOG.md b/CHANGELOG.md index 84085e8..6be6045 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] ### Added +- `scripts/seed-demo.py` fills a throwaway instance with a synthetic team — seven users, three models, 90 days of sessions and tool calls. It goes in over the OTLP endpoint rather than writing to DuckDB, so a seeded instance exercises the same ingest, cost-derivation and roll-up path a real one does, and it sends only attributes Claude Code actually sends: no `command` on `Bash` spans, so the Tools page shows the same "no command detail" state a real install sees. The RNG seed is fixed, so a re-run against a fresh volume reproduces the same numbers. `scripts/shoot-screenshots.mjs` turns that instance into the README images, each cropped at the bottom edge of a named element rather than at a pixel count ([docs/operations/screenshots.md](docs/operations/screenshots.md)) - `GET /api/v1/overview`, `/sessions`, `/costs` and `/models` accept `range`, the same five-key rolling window `/users` and `/tools` already took, and each echoes back the key it used. Defaults preserve today's behaviour instead of converging on one value: `/overview` and `/costs` default to `month` (the 30-day window they already applied), `/sessions` and `/models` to `all`, because they had no time filter and a `month` default would silently truncate every existing caller. Long ranges resolve against the `spans` ∪ `daily_usage` union at the raw-floor split, so `year` and `all` keep answering after retention has deleted the raw spans rather than repeating the `month` figure. On `/costs`, explicit `from`/`to` still beat the range key and the response then echoes `"range": null` ([ADR-0014](docs/decisions/0014-overview-single-range-selector.md)) - `GET /api/v1/sessions` returns `covered_since`. A session row needs a start time, model and status, none of which the roll-up keeps, so the list is raw-only; a range reaching past the raw floor is clamped and the field names the instant the list actually starts from (`null` when the range is fully covered). The Overview's Sessions block states that window in one line. The session *count* KPI is unaffected — `daily_usage` carries `session_id`, so counting distinct sessions across the union is exact @@ -18,6 +19,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - A failing `GET /api/v1/bash-commands` no longer renders as the "no command detail in this data" explainer. The Bash section branched on row count alone, so a request that errored looked identical to one that legitimately returned nothing, blaming Claude Code's telemetry for what was actually a server fault. Fetch failures now show the error ### Changed +- The README screenshots are of the software as it now is. The single hero shot predated the Overview rework — it showed the deleted user-search cloud, Sessions at the top, KPI labels hardcoded to `(30d)` and no range switcher. It is replaced by the current Overview plus a Screens gallery of Sessions, Costs, Users and Tools, all taken from the demo seed, so no real user name or real spend is published. The README's logo `` pointed at `assets/logo.svg`, a path that does not exist in the repo and rendered as a broken image on GitHub; it now points at `docs/assets/logo.svg`. The current-release tag reads `:0.3`, not `:0.2` - The Overview is one window instead of five. A single range switcher in the header — its own `cotel_overview_range` cookie, so it does not move the Users or Tools page — scopes every figure on the page. Previously the KPIs showed 30 days, the Sessions and Models blocks showed all time, and only the KPI labels said which, as a literal `(30d)` baked into the string; a reader comparing the Sessions KPI against the Models table below it was comparing 30 days against all time. Labels now take their suffix from the selected range, and `All` renders none - Overview section order is Users, History, Costs, Tools, Models, Sessions. A new Users block leads with the top 5 principals by spend in the selected range, and Sessions moves to the bottom as the one block that cannot honour a long range. The Costs block drops its inner by-model table — the Models block below it is the same data at full width - The Overview's user-search typeahead is gone, and the `UserSearch` component with it. Scoping is reached from a user's page ("View activity"); `?user_id=` now shows a chip in the header naming the user and clearing the scope on click, instead of a page that was silently filtered with nothing on it to say so diff --git a/README.md b/README.md index 9070c70..b5a18a5 100644 --- a/README.md +++ b/README.md @@ -1,10 +1,10 @@ -Flopsstuff logo +Flopsstuff logo # cotel — Claude Code Telemetry One Docker container. OTLP ingest on `:4318`, interactive analytics dashboard on `:8080`. No cloud dependencies, no sign-up. -![cotel dashboard](docs/assets/dashboard-screenshot.png) +![cotel Overview](docs/assets/dashboard-overview.png) ## What you get @@ -19,6 +19,15 @@ One Docker container. OTLP ingest on `:4318`, interactive analytics dashboard on - **Export / Import** — download all data as a versioned ZIP/CSV archive; restore it on a fresh instance - **Cloudflare Tunnel** — publish cotel over HTTPS with a single env var; bearer-token auth for OTLP, Zero Trust for the dashboard +## Screens + +Every screenshot here is one instance seeded with synthetic telemetry — see [Demo data](#demo-data). + +| | | +|:--|:--| +| **Sessions** — every session with its user, model, tokens, cost and OK / ERROR status
![Sessions](docs/assets/dashboard-sessions.png) | **Costs** — daily spend and cost by model across the window you pick
![Costs](docs/assets/dashboard-costs.png) | +| **Users** — cost and session count per principal for the selected range
![Users](docs/assets/dashboard-users.png) | **Tools** — call count, average duration and error rate per tool
![Tools](docs/assets/dashboard-tools.png) | + ## Quick start ```bash @@ -30,10 +39,27 @@ docker run -d \ ghcr.io/flopsstuff/cotel:latest ``` -> **Available tags:** `:latest` and `:0.2` (current release), `:0.x.y` (patch), `:main` (tip of main branch). +> **Available tags:** `:latest` and `:0.3` (current release), `:0.x.y` (patch), `:main` (tip of main branch). Open **http://localhost:8080** → **Setup** for the guided onboarding. +## Demo data + +An empty cotel shows empty charts, which makes it hard to judge. `scripts/seed-demo.py` +fills a throwaway instance with a synthetic team — seven users, three models, 90 days +of sessions and tool calls — over the real OTLP endpoint, so what you see is what +ingest actually produces: + +```bash +python3 scripts/seed-demo.py --dash-url http://localhost:8080 \ + --ingest-url http://localhost:4318 +``` + +It only creates users and ingests spans; it never deletes. Point it at an instance +you are willing to throw away, not at one holding real telemetry. The screenshots in +this README come from exactly this seed — the recipe is in +[docs/operations/screenshots.md](docs/operations/screenshots.md). + ## Point Claude Code at cotel Add to your `~/.claude/settings.json`: diff --git a/docs/.vitepress/config.js b/docs/.vitepress/config.js index e8a59a3..d4c47ad 100644 --- a/docs/.vitepress/config.js +++ b/docs/.vitepress/config.js @@ -30,6 +30,7 @@ export default defineConfig({ { text: 'Cloudflare Tunnel — Token Mode', link: '/operations/cloudflare-tunnel-remote' }, { text: 'Cloudflare Tunnel — Local Config', link: '/operations/cloudflare-tunnel-local' }, { text: 'Export / Import', link: '/operations/export-import' }, + { text: 'README Screenshots', link: '/operations/screenshots' }, ], }, ], diff --git a/docs/assets/dashboard-costs.png b/docs/assets/dashboard-costs.png new file mode 100644 index 0000000..872503c --- /dev/null +++ b/docs/assets/dashboard-costs.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f39a8c0af75fb3a30af41989f9cc2364a9b524fc6c974f45fd35f61f59846c47 +size 144381 diff --git a/docs/assets/dashboard-overview.png b/docs/assets/dashboard-overview.png new file mode 100644 index 0000000..4af29e0 --- /dev/null +++ b/docs/assets/dashboard-overview.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d79137ffa10ea44cc7812984ab259a7ad0b05e8bf685b679a9b23053aaba424c +size 235018 diff --git a/docs/assets/dashboard-screenshot.png b/docs/assets/dashboard-screenshot.png deleted file mode 100644 index 7365265..0000000 --- a/docs/assets/dashboard-screenshot.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:f6f3f9553fbbb45e01fd0e92fa984cbef7a0278a405ac18e101320a5f45f3705 -size 1192000 diff --git a/docs/assets/dashboard-sessions.png b/docs/assets/dashboard-sessions.png new file mode 100644 index 0000000..7b1d295 --- /dev/null +++ b/docs/assets/dashboard-sessions.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:d054178cc51787106610a66791ac3a529c4388235545b7b20b8a063be5168107 +size 356245 diff --git a/docs/assets/dashboard-tools.png b/docs/assets/dashboard-tools.png new file mode 100644 index 0000000..6095ac5 --- /dev/null +++ b/docs/assets/dashboard-tools.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:2cf41a8b6553a71e2f5e090f07dad841b05b147c693ae3080c2de0509cfa49e7 +size 146992 diff --git a/docs/assets/dashboard-users.png b/docs/assets/dashboard-users.png new file mode 100644 index 0000000..fb6f691 --- /dev/null +++ b/docs/assets/dashboard-users.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:dd9dae97d37d2a0e650584922849509adcf4ff1b05af8e1f919fbeb53cf440a5 +size 162194 diff --git a/docs/assets/logo.svg b/docs/assets/logo.svg new file mode 100644 index 0000000..c9a561e --- /dev/null +++ b/docs/assets/logo.svg @@ -0,0 +1,22 @@ + + + + + + + + + + + + + cotel + diff --git a/docs/operations/screenshots.md b/docs/operations/screenshots.md new file mode 100644 index 0000000..a52e0ab --- /dev/null +++ b/docs/operations/screenshots.md @@ -0,0 +1,66 @@ +# Re-taking the README screenshots + +The images in `docs/assets/dashboard-*.png` are produced from a throwaway instance +seeded with synthetic telemetry, never from a real one. Real telemetry carries real +user names and real spend, and a screenshot is forever. + +Redo them whenever a page in the shot changes shape. + +## 1. Build the image under test + +```bash +docker build -t cotel:shots . +``` + +## 2. Run it on a fresh volume, with retention off + +The seed reaches 90 days back; the shipped 30-day raw retention would roll most of +it into `daily_usage` mid-run and the session rows would vanish from the list. + +```bash +docker run -d --name cotel-shots \ + -p 14318:4318 -p 18080:8080 \ + -e COTEL_RETENTION_RAW_DAYS=3650 \ + -e COTEL_RETENTION_AGGREGATE_DAYS=3650 \ + -v cotel-shots-data:/data \ + cotel:shots +``` + +## 3. Seed it + +```bash +python3 scripts/seed-demo.py --dash-url http://localhost:18080 \ + --ingest-url http://localhost:14318 +``` + +Takes a couple of minutes and ingests ~25 000 spans across ~485 sessions. The RNG +seed is fixed, so a re-run against a fresh volume reproduces the same numbers. +Ingest is queued behind the HTTP response — wait for `span_count` in +`curl -s localhost:18080/api/v1/health` to stop climbing before shooting. + +## 4. Shoot + +```bash +BASE=http://localhost:18080 node scripts/shoot-screenshots.mjs +``` + +1440×1400 viewport at 2× DPR, dark scheme, each page cropped at the bottom edge of +a named element. The files land straight in `docs/assets/`. + +`playwright-core` has to be resolvable from the repo — `npx playwright-core@latest +--help` once is enough to populate the npx cache, then symlink or set `NODE_PATH` to +it. Chromium comes from `CHROMIUM` (default `/usr/bin/chromium`); this box has no +Playwright-managed browser. + +## 5. Tear down + +```bash +docker rm -f cotel-shots && docker volume rm cotel-shots-data +``` + +## What not to fake + +The seeder sends only the attributes Claude Code actually sends. In particular it +does not attach `command` to `Bash` spans, so the Tools page's Bash breakdown shows +its "no command detail in this data" state — which is what a real install sees. +Seeding it would make the README advertise a view nobody gets. diff --git a/scripts/seed-demo.py b/scripts/seed-demo.py new file mode 100755 index 0000000..855e019 --- /dev/null +++ b/scripts/seed-demo.py @@ -0,0 +1,243 @@ +#!/usr/bin/env python3 +"""Seed a cotel instance with synthetic Claude Code telemetry. + +Point it at a *throwaway* instance — it creates users and ingests spans, it +never deletes anything. Used to produce the README screenshots and to give a +fresh install something to look at. + +The payloads are the same OTLP/HTTP JSON Claude Code sends: a +``claude_code.model_invocation`` span per turn carrying model and token counts, +``claude_code.tool_use`` children carrying only ``tool_name``, and the session +id on the resource. Cost is left to cotel to derive, as it is in production. + + python3 scripts/seed-demo.py --dash-url http://localhost:8080 \ + --ingest-url http://localhost:4318 + +Run it against a database that already has data and you get two overlapping +datasets; start from an empty volume. +""" + +import argparse +import json +import random +import sys +import urllib.error +import urllib.request +from datetime import datetime, timedelta, timezone + +# Named principals, heaviest first. Weight drives how many sessions each one +# opens, so the Users top-5 has a readable spread instead of seven equal bars. +USERS = [ + ("alex-mbp", 1.00), + ("priya-mbp", 0.85), + ("sam-desktop", 0.70), + ("ci-runner", 0.60), + ("dana-wsl", 0.45), + ("jordan-mbp", 0.35), + ("nightly-bot", 0.25), +] + +# Per-user model mix: (model, weight). ci-runner and nightly-bot are automation +# and ride the cheap tiers. +MODEL_MIX = { + "ci-runner": [("claude-haiku-4-5", 0.7), ("claude-sonnet-5", 0.3)], + "nightly-bot": [("claude-haiku-4-5", 0.6), ("claude-sonnet-5", 0.4)], + "_default": [("claude-opus-5", 0.55), ("claude-sonnet-5", 0.35), ("claude-haiku-4-5", 0.10)], +} + +# (tool, share of calls, min ms, max ms, failure rate). A session is flagged +# ERROR when any one of its ~50 tool calls failed, so per-tool rates in the +# single-digit percent range would paint most of the sessions list red. +TOOLS = [ + ("Read", 0.26, 30, 400, 0.0008), + ("Edit", 0.17, 60, 900, 0.0042), + ("Bash", 0.16, 150, 24000, 0.0086), + ("Grep", 0.12, 80, 1200, 0.0012), + ("Glob", 0.08, 40, 600, 0.0006), + ("Write", 0.07, 50, 700, 0.0016), + ("TodoWrite", 0.06, 20, 120, 0.0), + ("Task", 0.04, 18000, 240000, 0.0034), + ("WebFetch", 0.03, 900, 6500, 0.0148), + ("NotebookEdit", 0.01, 80, 800, 0.0020), +] + +# Weekday index -> session multiplier. Weekends are quiet but not dead. +DAY_WEIGHT = [1.0, 1.05, 1.0, 0.95, 0.8, 0.22, 0.18] + +STATUS_OK = 1 +STATUS_ERROR = 2 + + +def post(url, payload, token=None, timeout=60): + body = json.dumps(payload).encode() + req = urllib.request.Request(url, data=body, method="POST") + req.add_header("Content-Type", "application/json") + if token: + req.add_header("Authorization", "Bearer " + token) + with urllib.request.urlopen(req, timeout=timeout) as resp: + raw = resp.read() + return json.loads(raw) if raw else {} + + +def create_users(dash_url, rng): + tokens = {} + for name, _ in USERS: + try: + created = post(dash_url + "/api/v1/users", {"name": name}) + except urllib.error.HTTPError as e: + sys.exit("could not create user %s: %s %s" % (name, e.code, e.read().decode()[:200])) + tokens[name] = created["token"] + return tokens + + +def pick(rng, weighted): + total = sum(w for _, w in weighted) + r = rng.random() * total + for item, w in weighted: + r -= w + if r <= 0: + return item + return weighted[-1][0] + + +def hex_id(rng, n): + return "".join(rng.choice("0123456789abcdef") for _ in range(n)) + + +def nano(ts): + return str(int(ts.timestamp() * 1_000_000_000)) + + +def model_tokens(rng, model): + """Token counts for one turn. Cache reads dominate, as they do in real use.""" + if model.startswith("claude-haiku"): + scale = 0.45 + elif model.startswith("claude-sonnet"): + scale = 0.75 + else: + scale = 1.0 + return { + "input_tokens": int(rng.randint(400, 3200) * scale), + "output_tokens": int(rng.randint(180, 3600) * scale), + "cache_read_tokens": int(rng.randint(18000, 240000) * scale), + "cache_creation_tokens": int(rng.randint(0, 9000) * scale), + } + + +def build_session(rng, user, start, session_id): + """Return the OTLP resourceSpans payload for one session.""" + model = pick(rng, MODEL_MIX.get(user, MODEL_MIX["_default"])) + trace_id = hex_id(rng, 32) + turns = rng.randint(4, 22) + # A session that errors does so on a tool, not on the whole run; roughly one + # in ten ends up flagged ERROR on the sessions list. + spans = [] + cursor = start + + for _ in range(turns): + think = timedelta(seconds=rng.randint(4, 70)) + inv_start = cursor + inv_end = inv_start + think + attrs = [{"key": "model", "value": {"stringValue": model}}] + for key, val in model_tokens(rng, model).items(): + attrs.append({"key": key, "value": {"intValue": str(val)}}) + parent = hex_id(rng, 16) + spans.append({ + "traceId": trace_id, + "spanId": parent, + "parentSpanId": "", + "name": "claude_code.model_invocation", + "startTimeUnixNano": nano(inv_start), + "endTimeUnixNano": nano(inv_end), + "status": {"code": STATUS_OK}, + "attributes": attrs, + }) + cursor = inv_end + + for _ in range(rng.randint(1, 5)): + tool, _share, lo, hi, fail_rate = pick( + rng, [(t, t[1]) for t in TOOLS] + ) + dur = timedelta(milliseconds=rng.randint(lo, hi)) + failed = rng.random() < fail_rate + spans.append({ + "traceId": trace_id, + "spanId": hex_id(rng, 16), + "parentSpanId": parent, + "name": "claude_code.tool_use", + "startTimeUnixNano": nano(cursor), + "endTimeUnixNano": nano(cursor + dur), + "status": {"code": STATUS_ERROR if failed else STATUS_OK}, + "attributes": [{"key": "tool_name", "value": {"stringValue": tool}}], + }) + cursor += dur + timedelta(milliseconds=rng.randint(200, 2500)) + + return { + "resourceSpans": [{ + "resource": { + "attributes": [ + {"key": "service.name", "value": {"stringValue": "claude-code"}}, + {"key": "service.version", "value": {"stringValue": "2.1.4"}}, + {"key": "session.id", "value": {"stringValue": session_id}}, + ] + }, + "scopeSpans": [{ + "scope": {"name": "claude.code", "version": "2.1.4"}, + "spans": spans, + }], + }] + } + + +def main(): + ap = argparse.ArgumentParser(description=__doc__) + ap.add_argument("--dash-url", default="http://localhost:8080") + ap.add_argument("--ingest-url", default="http://localhost:4318") + ap.add_argument("--days", type=int, default=90) + ap.add_argument("--seed", type=int, default=1729, help="RNG seed; fixed so runs are reproducible") + args = ap.parse_args() + + dash = args.dash_url.rstrip("/") + ingest = args.ingest_url.rstrip("/") + "/v1/traces" + rng = random.Random(args.seed) + + tokens = create_users(dash, rng) + print("created %d users" % len(tokens)) + + now = datetime.now(timezone.utc) + sessions = 0 + spans = 0 + + for day_offset in range(args.days, -1, -1): + day = (now - timedelta(days=day_offset)).replace( + hour=0, minute=0, second=0, microsecond=0 + ) + weight = DAY_WEIGHT[day.weekday()] + for user, user_weight in USERS: + count = rng.random() * 3.4 * weight * user_weight + for _ in range(int(count) + (1 if rng.random() < count % 1 else 0)): + # Working hours, jittered; the current day stops at "now" so the + # newest rows are not in the future. + start = day + timedelta( + hours=rng.randint(8, 20), + minutes=rng.randint(0, 59), + seconds=rng.randint(0, 59), + ) + if start > now - timedelta(minutes=25): + continue + session_id = "sess_" + hex_id(rng, 12) + payload = build_session(rng, user, start, session_id) + try: + post(ingest, payload, token=tokens[user]) + except urllib.error.URLError as e: + sys.exit("ingest failed: %s" % e) + sessions += 1 + spans += len(payload["resourceSpans"][0]["scopeSpans"][0]["spans"]) + if day_offset % 10 == 0: + print(" ...%s: %d sessions, %d spans so far" % (day.date(), sessions, spans)) + + print("seeded %d sessions / %d spans across %d days" % (sessions, spans, args.days)) + + +if __name__ == "__main__": + main() diff --git a/scripts/shoot-screenshots.mjs b/scripts/shoot-screenshots.mjs new file mode 100644 index 0000000..e913878 --- /dev/null +++ b/scripts/shoot-screenshots.mjs @@ -0,0 +1,61 @@ +// Re-take the README screenshots against a seeded throwaway instance. +// +// npx playwright-core@latest --help >/dev/null # once, to populate the cache +// BASE=http://localhost:8080 node scripts/shoot-screenshots.mjs +// +// Needs playwright-core resolvable from here and a chromium at CHROMIUM (default +// /usr/bin/chromium). Full recipe: docs/operations/screenshots.md. + +import { chromium } from 'playwright-core' + +const BASE = process.env.BASE || 'http://localhost:8080' +const OUT = process.env.OUT || 'docs/assets' +const CHROMIUM = process.env.CHROMIUM || '/usr/bin/chromium' + +// Each shot is cut at the bottom of a real element rather than at a pixel +// count, so a row height or chart size change does not slice a card in half. +const STAT_SECTIONS = `[...document.querySelectorAll('div')].filter(d => typeof d.className === 'string' && /card/i.test(d.className) && d.querySelector(':scope > div > button[aria-expanded]'))` + +const SHOTS = [ + { name: 'dashboard-overview', path: '/', wait: '.recharts-surface', end: `(${STAT_SECTIONS})[1]` }, + { name: 'dashboard-users', path: '/users', wait: 'table', end: `document.querySelector('table')` }, + { name: 'dashboard-tools', path: '/tools', wait: 'table', end: `document.querySelector('table')` }, + { name: 'dashboard-sessions', path: '/sessions', wait: 'table', end: `document.querySelectorAll('tbody tr')[8]` }, + { + name: 'dashboard-costs', + path: '/costs', + wait: '.recharts-surface', + end: `[...document.querySelectorAll('div')].find(d => typeof d.className === 'string' && /card/i.test(d.className) && /^cost by model/i.test(d.textContent.trim()))`, + }, +] + +const WIDTH = 1440 +const MAX_HEIGHT = 1400 +const PAD = 8 + +const browser = await chromium.launch({ executablePath: CHROMIUM }) +const ctx = await browser.newContext({ + viewport: { width: WIDTH, height: MAX_HEIGHT }, + deviceScaleFactor: 2, + colorScheme: 'dark', +}) +const page = await ctx.newPage() + +for (const shot of SHOTS) { + await page.goto(BASE + shot.path, { waitUntil: 'networkidle' }) + await page.waitForSelector(shot.wait, { timeout: 15000 }) + await page.waitForTimeout(2500) // chart entry animations + + const bottom = await page.evaluate(`(() => { const el = ${shot.end}; if (!el) return null; + const r = el.getBoundingClientRect(); return Math.ceil(r.bottom + window.scrollY) })()`) + if (!bottom) throw new Error(`${shot.name}: could not locate the element to crop at`) + + const height = Math.min(bottom + PAD, MAX_HEIGHT) + await page.screenshot({ + path: `${OUT}/${shot.name}.png`, + clip: { x: 0, y: 0, width: WIDTH, height }, + }) + console.log(`${shot.name}.png (${WIDTH}x${height} @2x)`) +} + +await browser.close()