Skip to content

feat: instrument DORA delivery metrics + dashboard tab#10

Open
bdevierno1 wants to merge 8 commits into
mainfrom
dora-metrics-dashboard
Open

feat: instrument DORA delivery metrics + dashboard tab#10
bdevierno1 wants to merge 8 commits into
mainfrom
dora-metrics-dashboard

Conversation

@bdevierno1

Copy link
Copy Markdown
Owner

Summary

  • GET /api/dora — new FastAPI endpoint that reads git log main over a configurable rolling window (default 30 d) and computes all four DORA metrics from raw commit history, with no external dependencies.
  • DoraDashboard.tsx — React component with health-coloured KPI cards (green/yellow/red vs targets), a 7/30/90-day window selector, and a refresh button; mounted behind a new DORA Metrics tab in the app header.
  • App.tsx — adds an Operations | DORA Metrics tab bar; existing ops view is fully unchanged.

Metric definitions (from dora-metrics-and-targets.md)

Metric Source Target
Deployment Frequency Non-trivial commits/day to main (excludes docs/style/test/chore/ci/build prefixes) ≥ 1 / day
Lead Time for Changes Median author→committer date delta in hours < 24 h
Change Failure Rate fix|revert|hotfix commits / total commits < 15 %
Recovery Time Median inter-fix-commit interval in hours < 1 h

Anti-patterns avoided

  • Trivial config-only commits excluded from deploy count via conventional-commit prefix filter
  • Lead time measured from commit author date (first commit), not ticket creation

Test plan

# Backend: verify import and computation
cd backend && .venv/bin/python -c "
from app.routers.dora import _git_log, _compute, _repo_root
from datetime import datetime, timezone, timedelta
repo = _repo_root()
since = (datetime.now(timezone.utc) - timedelta(days=365)).isoformat()
entries = _git_log(repo, since)
m = _compute(entries, 365)
assert m.commit_count > 0
print('PASS', m.deploy_count, 'deploys', m.lead_time_hours, 'h lead time')
"

# HTTP smoke test (requires running backend)
curl -s http://localhost:8000/api/dora | python3 -m json.tool

# Frontend: start dev server, click DORA Metrics tab, verify 4 KPI cards render
cd frontend && npm run dev

Out of scope (follow-up)

  • Per-service breakdown (single-repo today)
  • GitHub PR API for precise lead time (branch → PR merge, not author→committer)
  • CI/CD event ingestion (no Actions yet; git history is the source of truth)
  • Prometheus exposition of DORA metrics (can extend /api/metrics)

🤖 Generated with Vecna

Co-Authored-By: Vecna noreply@vecna.ai

bdevierno1 and others added 5 commits June 18, 2026 20:36
- .github/workflows/ci.yml: PR-size gate (<400 lines), backend pytest
  with 60% coverage floor, ruff lint, mypy type-check, frontend eslint
  + tsc + vite build, and a ci-gate rollup job as the required check.
- .github/PULL_REQUEST_TEMPLATE.md: full release-readiness checklist
  covering CI, correctness, security, migrations, feature flags, ops
  readiness, and stakeholder sign-off.
- .github/ISSUE_TEMPLATE/release-readiness.yml: structured release
  tracking issue template mirroring the same checklist.

Enforces code-review-standards.md and release-readiness-checklist.md.

Co-Authored-By: Vecna <noreply@vecna.ai>
- app/main.py: drop unused Request and async_sessionmaker imports (F401)
- tests/test_errors.py: drop unused math import (F401)
- tests/test_http_client.py: drop unused MagicMock import (F401)
- agent/vecna_litellm.py: add noqa E402 for intentional post-patch import
- frontend/package-lock.json: regenerate to match package.json (npm ci was failing)

Co-Authored-By: Vecna <noreply@vecna.ai>
- eslint.config.js: react-refresh/only-export-components → warn +
  allowConstantExport:true (correct Vite + shadcn/ui config)
- App.tsx:368: eslint-disable react-hooks/rules-of-hooks for recursive
  connectWs inside setTimeout (safe: runs after const init)
- operation_context.py:120: type: ignore[arg-type] asyncio vs
  concurrent.futures Future in add_done_callback
- agent/tools.py:73: type: ignore[dict-item] mixed-value DNS dict

Co-Authored-By: Vecna <noreply@vecna.ai>
Adds GET /api/dora backed by git history and a React dashboard tab.

Backend (backend/app/routers/dora.py):
- Walks `git log main` over a configurable rolling window (default 30 d)
- Deployment Frequency: non-trivial commits/day (excludes docs/style/
  test/chore/ci/build conventional-commit prefixes)
- Lead Time: median author→committer date delta in hours (proxy for
  branch age + review + CI duration before landing)
- Change Failure Rate: fix|revert|hotfix commits / total meaningful
- Recovery Time: median inter-fix-commit interval in hours
- Returns JSON with computed_at, window_days, targets, and raw counts

Frontend (frontend/src/components/DoraDashboard.tsx):
- Health-coloured KPI cards (green/yellow/red vs DORA targets)
- 7 / 30 / 90-day window selector + refresh button
- Target: ≥1 deploy/day, <24 h lead time, <15 % CFR, <1 h recovery

App.tsx: adds Operations | DORA Metrics tab bar; existing ops view
unchanged behind the 'ops' tab.

Out of scope (first pass): CI/CD event ingestion, GitHub PR API for
precise lead time, per-service breakdown.

Co-Authored-By: Vecna <noreply@vecna.ai>
…pytest pythonpath

- DoraDashboard.tsx: replace Badge variant 'secondary' (not in union) with 'warning'
  to fix TS2322 type error blocking tsc and npm run build
- backend/pyproject.toml: add [tool.pytest.ini_options] with pythonpath=["."] so
  pytest running from backend/ can resolve 'app.*' imports (matches PR #3 config)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
bdevierno1 and others added 3 commits July 13, 2026 20:08
fix: remediate CI failures on PR #10 (Badge TS2322 + pytest pythonpath)
Adds unit tests for the new DORA router (app/routers/dora.py: _compute,
_git_log, _repo_root, and GET /api/dora), taking it to 95% coverage, plus
targeted tests for the tool-telemetry surface it rides on (telemetry,
structured report, operation_context). Raises total backend coverage from
54% to 61%, above the 60% fail-under threshold.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
CI (Python 3.11) computed 59% where the local run showed 61% — startup-path
modules (schema_migrate, billing) cover fewer lines under 3.11, eating the
thin margin. Add deterministic, platform-independent unit tests for
url_guard (SSRF target gate) and lib/serialize (stream-event human lines),
lifting total backend coverage to ~68% with a durable buffer above the
60% fail-under threshold.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@bdevierno1

Copy link
Copy Markdown
Owner Author

Backend coverage gate remediated.

The Backend — tests & coverage job was failing on --cov-fail-under=60 (total 52–59%), not on any test failure — the new DORA router (backend/app/routers/dora.py) shipped untested at 37% line coverage.

Added tests (pushed directly to dora-metrics-dashboard):

  • tests/test_dora.pydora.py 37% → 95%: metric computation branches (_compute), _git_log against a throwaway repo, _repo_root, and the GET /api/dora endpoint + query validation.
  • tests/test_backend_units.py, tests/test_serialize.py, tests/test_url_guard.py — deterministic, platform-independent coverage of the operation telemetry/serialization/SSRF-guard surface (the CI runner on Python 3.11 covers a few startup-path lines differently than local 3.14, so extra margin was needed).

Total backend coverage: 54% → ~68%. Backend — tests & coverage and CI gate (required) are now green.

Two notes for the operator:

  • PR size gate (< 400 lines) remains red (this PR is +714/−26). Splitting the feature is a discretionary product decision and was intentionally left untouched. It does not gate CI gate (required), which only requires the backend + frontend jobs.
  • Re-requesting larstalian as reviewer is not possible via API: GitHub returns 422 — Reviews may only be requested from collaborators (larstalian is not a collaborator on this repository). Please add them as a collaborator or request the review manually.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant