Local report viewer, themed sortable reports, axe-core e2e checks

CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## Unreleased
+
+- Add `evalarc view [DIR]`: a read-only, loopback-only local viewer that lists
+  every report under a folder with its verdict, following the `inspect view` /
+  `promptfoo view` pattern with the standard library only. It rejects foreign
+  `Host` headers, path traversal and directory listings. `--write-index` writes
+  the same index as a static page.
+- Report pages: light and dark themes from the system setting, sticky table
+  headers, tabular numbers, sortable columns (WAI-ARIA APG pattern) and a row
+  filter on long tables. The enhancement script is a packaged asset inlined and
+  allowed by CSP hash; pages remain complete without JavaScript.
+- `scripts/check_reports_browser.cjs` now uses axe-core (WCAG 2.2 A/AA, light and
+  dark) instead of a custom contrast check, and runs end-to-end reviewer tasks:
+  verdict on the first screen, skip link, keyboard sorting, filtering, 320 px
+  reflow, no-JavaScript, print, and browsing from the viewer index.
+  `@axe-core/playwright` 4.13.0 is a pinned dev dependency.
+
 ## 0.17.3 — 2026-10-01
 
 - fix: fail the diff gate when the baseline run did not finish. evalarc diff and the GitHub Action now fail the gate (exit 1) when the baseline run did not finish. Before this change, an Inspect baseline log with a status other than success could pass, because checks missing from it were classified as 'added', which never blocks. diff.json now records "baseline_incomplete": true. The key is omitted for a finished baseline, so those reports are unchanged apart from created_at and version. Default stdout prints 'The baseline run is incomplete (status error); N check(s) appear only in the current run and were not compared. Rerun the baseline to completion; the gate fails.' summary.md always includes the line below the table: 'The baseline run did not finish (status `error`); the gate fails because checks missing from it cannot be compared. N check(s) appear only in the current run. Rerun the baseline to completion.' Its heading says 'the baseline run is incomplete' only when there are no blocking changes and the current run finished; otherwise the existing blocking or both-incomplete heading appears. index.html shows 'Gate failed' with the reason 'the baseline run is incomplete'. When no blocking rows exist, it tells you to rerun the baseline. evalarc verify recomputes the same result. docs/ci-gate.md and the README CI-gate paragraph document the rule.

README.md

@@ -92,6 +92,7 @@ to produce your first HTML report without cloning the source.
 | A cheaper model, lower thinking effort or shorter prompt | Require no lost checks and recorded cost or tokens at most a set ratio of the baseline | [Cost at equal quality](docs/ci-gate.md#cost-at-equal-quality) |
 | Saved results you plan to tune against | Find saturation, never-passing and flaky checks, inconsistent grading, truncation, self-grading, pipeline errors and noise above the change you need | [Eval health](docs/eval-health.md) |
 | A grader you are about to trust, or a change judged by preference | Blind spot checks with false accepts/rejects; blind randomized A/B against the baseline with position-bias checks | [Blind judging](docs/judging.md) |
+| A folder of reports from many runs | Open one local index with every report's verdict; sort, filter and drill in | [Browse reports](docs/viewer.md) |
 | A new evaluation for your application | Scaffold cases, a programmatic grader, a runner and a hillclimb config; review the inputs before any run | [Evaluation project](docs/eval-init.md) |
 | A prompt or skill you want to hillclimb with your own model-calling commands | Run the loop: tuning failures only, edits limited to allowed files, pasted-case and leakage rollback, keep/rollback by held-out results | [Hillclimb run](docs/hillclimb-run.md) |
 | Results of each step of a hillclimbing loop | Replay keep/rollback rules on tuning and held-out cases, triage stalls, and decide whether to merge | [Hillclimb review](docs/hillclimb-review.md) |

README.zh-CN.md

@@ -77,6 +77,7 @@ PyPI 与 GitHub Releases 提供相同的 wheel 和源码包，也支持
 | 更便宜的模型、更低思考强度或更短的提示词 | 要求不丢失检查项，且记录的成本或 token 不超过基线的设定倍数 | [保质降本](docs/ci-gate.zh-CN.md#保质降本) |
 | 准备据以调优的已保存结果 | 发现饱和、始终失败或波动的检查、评分不一致、截断、自评、流水线错误，以及大于目标效应的噪声 | [评测健康检查](docs/eval-health.zh-CN.md) |
 | 准备采信的评分器，或按偏好评判的改动 | 盲抽查并统计误判通过/失败；与基线做随机 A/B 盲评并检查位置偏差 | [盲评](docs/judging.zh-CN.md) |
+| 多次运行产生的一批报告 | 用一个本地索引查看每份报告的结论，可排序、筛选并逐个打开 | [浏览报告](docs/viewer.zh-CN.md) |
 | 为自己的应用新建评测 | 生成用例、程序化评分器、评测脚本与调优配置；运行前审查输入 | [创建评测项目](docs/eval-init.zh-CN.md) |
 | 想用自己的模型调用命令做爬山调优的提示词或技能 | 运行循环：只给调优失败、只许改允许的文件、粘贴用例与泄露自动回滚、按留出结果保留或回滚 | [运行爬山调优](docs/hillclimb-run.zh-CN.md) |
 | 爬山调优每一步的结果 | 按调优与留出用例重放保留/回滚规则，分类停滞原因，判断是否合并 | [爬山调优复核](docs/hillclimb-review.zh-CN.md) |

SECURITY.md

@@ -46,6 +46,10 @@ model or network calls. The hillclimb `allow` list compares the workspace (inclu
 on any other change; it is a guard, not a sandbox, and cannot see or undo changes
 outside the workspace.
 
+`evalarc view` serves report folders read-only on `127.0.0.1` only, rejects other
+`Host` headers, path traversal and directory listings, and has no API. Report pages
+allow exactly one inline script by CSP hash and no network access.
+
 Local mode requires `--trust-local`. It executes with the host user's filesystem
 and network privileges. Clearing the child environment does not provide a
 sandbox. Use local mode only for code you trust, such as the bundled audit

docs/viewer.md

@@ -0,0 +1,50 @@
+# Browse reports
+
+Every EvalArc command that takes `--output` writes a self-contained HTML report.
+`evalarc view` lists all of them under a folder with their verdicts and opens it
+in your browser. [中文](viewer.zh-CN.md).
+
+```bash
+evalarc view runs            # http://127.0.0.1:7576/, opens a browser
+evalarc view runs --port 8000 --no-browser
+evalarc view runs --write-index site/index.html   # static index for CI artifacts or Pages
+```
+
+The index shows each report folder (newest first) with its verdict (✓ pass,
+✗ fail, ! check), kind and summary, read from the verdict banner the report
+already contains. It sorts and filters like every report table.
+
+## Why a local viewer, not a desktop app
+
+Evaluation tools converge on the same pattern: a read-only local web server plus
+the browser you already have (`inspect view`, `promptfoo view`, `mlflow ui`),
+with an IDE extension where tighter integration is needed (Inspect's VS Code
+extension), and a static bundle for sharing (`inspect view bundle`,
+`promptfoo eval -o report.html`). A desktop shell (Electron, Tauri) would add a
+100 MB+ runtime or a Rust toolchain for no capability the browser lacks, and
+would break EvalArc's no-runtime-dependency install. `evalarc view` uses the
+Python standard library only.
+
+## Security
+
+The viewer binds to `127.0.0.1` only and serves files inside the given folder:
+it rejects requests whose `Host` is not `127.0.0.1:PORT` or `localhost:PORT`
+(DNS-rebinding protection), path traversal, symlinks leaving the folder and
+directory listings. It is read-only and has no API. To view reports on a remote
+machine, forward the port over SSH instead of exposing it:
+
+```bash
+ssh -L 7576:127.0.0.1:7576 user@remote   # then run evalarc view there
+```
+
+## Report pages
+
+Reports open with the decision (verdict, reason, next step) and work in light
+and dark mode, on phones, at 400% zoom, in print and without JavaScript. With
+JavaScript, table headers become sort buttons (WAI-ARIA APG sortable-table
+pattern) and tables with eight or more rows get a labelled filter. The script is
+inlined and allowed by a SHA-256 hash in each page's Content-Security-Policy; no
+other script, network request or remote asset is permitted.
+`scripts/check_reports_browser.cjs` checks every report kind and the viewer with
+axe-core (WCAG 2.2 A/AA) and end-to-end reviewer tasks in CI. Automated checks
+do not replace testing with a screen reader.

docs/viewer.zh-CN.md

@@ -0,0 +1,27 @@
+# 浏览报告
+
+所有带 `--output` 的 EvalArc 命令都会写出自包含的 HTML 报告。`evalarc view` 列出某目录下的
+全部报告及其结论，并在浏览器中打开。[English](viewer.md)。
+
+```bash
+evalarc view runs            # http://127.0.0.1:7576/，自动打开浏览器
+evalarc view runs --port 8000 --no-browser
+evalarc view runs --write-index site/index.html   # 为 CI 产物或 Pages 生成静态索引
+```
+
+索引按时间倒序列出每个报告目录的结论（✓ 通过、✗ 失败、! 需检查）、类型与摘要，可排序和筛选。
+
+**为什么是本地查看器而非桌面客户端：** 主流评测工具都采用只读本地 Web 服务加用户已有浏览器的
+方式（`inspect view`、`promptfoo view`、`mlflow ui`），需要深度集成时提供 IDE 插件，分享时
+生成静态包。Electron/Tauri 桌面壳会带来 100 MB 以上运行时或 Rust 工具链，却不提供浏览器没有
+的能力，还会破坏 EvalArc 零运行时依赖的安装方式。`evalarc view` 只用 Python 标准库。
+
+**安全：** 只绑定 `127.0.0.1`，只提供该目录内的文件；拒绝非 `127.0.0.1:端口`/`localhost:端口`
+的 Host（防 DNS 重绑定）、路径穿越、指向目录外的符号链接和目录列表；只读、无 API。远程机器请用
+`ssh -L 7576:127.0.0.1:7576 user@remote` 转发端口，不要直接暴露。
+
+**报告页面：** 先给结论（结果、原因、下一步），支持深浅色、手机、400% 缩放、打印和禁用 JavaScript。
+启用 JavaScript 时，表头变为排序按钮（WAI-ARIA APG 可排序表格模式），8 行以上的表格带筛选框。
+脚本内联并由页面 CSP 中的 SHA-256 哈希放行，不允许其他脚本、网络请求或远程资源。
+`scripts/check_reports_browser.cjs` 在 CI 中用 axe-core（WCAG 2.2 A/AA）和端到端任务检查全部报告
+类型与查看器；自动检查不能替代屏幕阅读器实测。

package.json

@@ -7,6 +7,7 @@
     "test:site": "node scripts/check_site.cjs"
   },
   "devDependencies": {
+    "@axe-core/playwright": "4.13.0",
     "playwright": "1.63.0"
   }
 }