Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,8 @@ jobs:
- run: python scripts/build_dataset.py --output dist/casebook
- run: python scripts/swe_dataset.py --output dist/independent-swe-dataset
- run: npm ci
- name: The committed report script matches its TypeScript source
run: npm run check:frontend
- run: npx playwright install --with-deps chromium
- run: npm run test:site
- run: node scripts/check_model_upgrade.cjs
Expand Down
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@
verdict on the first screen, skip link, keyboard sorting, filtering, 320 px
reflow, no-JavaScript, print, and browsing from the viewer index.
`@axe-core/playwright` 4.13.0 is a pinned dev dependency.
- The report front end is now TypeScript (`frontend/report_enhance.ts`, strict
mode), compiled to the packaged `report_enhance.js`. The compiled file is
committed, so installing EvalArc still needs no Node; CI fails when it is
stale. TypeScript 7.0.2 is a pinned dev dependency. The Python core is
unchanged.

## 0.17.3 — 2026-10-01

Expand Down
4 changes: 3 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
Install with `python -m pip install -e ".[dev]"`, then run `pytest -q`,
`ruff check .`, and `ruff format --check .`. Install Node.js to exercise the
independent JavaScript support policy; that test is skipped if Node is absent.
CI explicitly provisions Node. See [task authoring](docs/task-authoring.md) for
CI explicitly provisions Node. Report browser code is TypeScript in `frontend/`:
after editing it, run `npm ci`, `npm run build:frontend` and `npm run test:reports`,
and commit the regenerated `src/evalarc/assets/report_enhance.js`. See [task authoring](docs/task-authoring.md) for
the current built-in registry and evidence contract.

The [workflow guide](docs/workflow.md) covers readiness checks and comparisons.
Expand Down
1 change: 1 addition & 0 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -16,3 +16,4 @@ recursive-include site *.html *.css *.js *.svg
recursive-include scripts *.py *.cjs *.html *.md
recursive-include huggingface *.md
include package.json package-lock.json
recursive-include frontend *.ts *.json
10 changes: 10 additions & 0 deletions docs/viewer.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,16 @@ machine, forward the port over SSH instead of exposing it:
ssh -L 7576:127.0.0.1:7576 user@remote # then run evalarc view there
```

## Architecture

The core (parsing, gates, verification, the viewer server) is Python with no
runtime dependencies, like Inspect AI. The browser code is TypeScript in
`frontend/`, type-checked in strict mode and compiled to
`src/evalarc/assets/report_enhance.js`. The compiled file is committed so
`pip install evalarc` never needs Node; `npm run check:frontend` fails in CI if
it differs from a fresh compile. After editing `frontend/*.ts`, run
`npm run build:frontend` and commit both files.

## Report pages

Reports open with the decision (verdict, reason, next step) and work in light
Expand Down
5 changes: 5 additions & 0 deletions docs/viewer.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,11 @@ evalarc view runs --write-index site/index.html # 为 CI 产物或 Pages 生
的 Host(防 DNS 重绑定)、路径穿越、指向目录外的符号链接和目录列表;只读、无 API。远程机器请用
`ssh -L 7576:127.0.0.1:7576 user@remote` 转发端口,不要直接暴露。

**架构:** 核心(解析、门禁、复核、查看器服务)是零运行时依赖的 Python,与 Inspect AI 相同;浏览器端
代码是 `frontend/` 下的 TypeScript,严格模式类型检查后编译为 `src/evalarc/assets/report_enhance.js`。
编译产物随仓库提交,因此 `pip install evalarc` 不需要 Node;若它与重新编译的结果不一致,CI 中的
`npm run check:frontend` 会失败。修改 `frontend/*.ts` 后请运行 `npm run build:frontend` 并提交两者。

**报告页面:** 先给结论(结果、原因、下一步),支持深浅色、手机、400% 缩放、打印和禁用 JavaScript。
启用 JavaScript 时,表头变为排序按钮(WAI-ARIA APG 可排序表格模式),8 行以上的表格带筛选框。
脚本内联并由页面 CSP 中的 SHA-256 哈希放行,不允许其他脚本、网络请求或远程资源。
Expand Down
111 changes: 111 additions & 0 deletions frontend/report_enhance.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
// Progressive enhancement for EvalArc offline reports, compiled to
// src/evalarc/assets/report_enhance.js (committed, so `pip install` needs no Node).
// The Python core inlines the compiled file and allows it by CSP hash; without
// JavaScript every table still renders in full. Sortable columns follow the WAI-ARIA
// APG sortable-table pattern (a button inside each header, aria-sort on the sorted
// column only). Tables with 8+ rows get a labelled row filter with a live count.
// Tables marked data-static are left alone.

(() => {
type Direction = "ascending" | "descending";

const SORTABLE_MIN_ROWS = 3;
const FILTER_MIN_ROWS = 8;

function leadingNumber(text: string): number {
const match = text.replace(/,/g, "").match(/[-+]?\d*\.?\d+/);
return match ? parseFloat(match[0]) : NaN;
}

function cellText(row: HTMLTableRowElement, column: number): string {
const cell = row.cells[column];
return cell ? (cell.textContent ?? "").trim() : "";
}

function compare(a: string, b: string): number {
const na = leadingNumber(a);
const nb = leadingNumber(b);
if (!Number.isNaN(na) && !Number.isNaN(nb)) return na - nb;
return a.localeCompare(b, undefined, { numeric: true });
}

function makeSortable(body: HTMLTableSectionElement, headers: HTMLTableCellElement[]): void {
headers.forEach((th, column) => {
const label = (th.textContent ?? "").trim();
if (!label) return;
const button = document.createElement("button");
button.type = "button";
const icon = document.createElement("span");
icon.setAttribute("aria-hidden", "true");
icon.textContent = "↕";
button.append(document.createTextNode(`${label} `), icon);
th.textContent = "";
th.append(button);
button.addEventListener("click", () => {
const direction: Direction =
th.getAttribute("aria-sort") === "ascending" ? "descending" : "ascending";
for (const other of headers) {
other.removeAttribute("aria-sort");
const mark = other.querySelector<HTMLSpanElement>("button span");
if (mark) mark.textContent = "↕";
}
th.setAttribute("aria-sort", direction);
icon.textContent = direction === "ascending" ? "▲" : "▼";
const sorted = Array.from(body.rows).sort((x, y) => {
const order = compare(cellText(x, column), cellText(y, column));
return direction === "ascending" ? order : -order;
});
for (const row of sorted) body.append(row);
});
});
}

function addFilter(table: HTMLTableElement, rows: HTMLTableRowElement[], index: number): void {
const id = `filter-${index}`;
const tools = document.createElement("div");
tools.className = "tools";
const label = document.createElement("label");
label.htmlFor = id;
label.textContent = "Filter rows";
const input = document.createElement("input");
input.type = "search";
input.id = id;
input.autocomplete = "off";
const count = document.createElement("span");
count.className = "count";
count.setAttribute("aria-live", "polite");
const update = (): void => {
const query = input.value.trim().toLowerCase();
let shown = 0;
for (const row of rows) {
const hit = !query || (row.textContent ?? "").toLowerCase().includes(query);
row.hidden = !hit;
if (hit) shown += 1;
}
count.textContent = `${shown} of ${rows.length} rows`;
};
input.addEventListener("input", update);
tools.append(label, input, count);
(table.closest(".scroll") ?? table).before(tools);
update();
}

function enhance(): void {
let filters = 0;
document.querySelectorAll<HTMLTableElement>("main table").forEach((table) => {
const body = table.tBodies[0];
const head = table.tHead;
if (!body || !head || !head.rows[0]) return;
if (body.rows.length < SORTABLE_MIN_ROWS || table.dataset.static !== undefined) return;
const rows = Array.from(body.rows);
makeSortable(body, Array.from(head.rows[0].cells));
if (rows.length >= FILTER_MIN_ROWS) {
filters += 1;
addFilter(table, rows, filters);
}
});
}


enhance();
})();
19 changes: 19 additions & 0 deletions frontend/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"compilerOptions": {
"target": "ES2020",
"lib": ["ES2020", "DOM", "DOM.Iterable"],
"module": "preserve",
"moduleDetection": "legacy",
"strict": true,
"noUnusedLocals": true,
"noUnusedParameters": true,
"noImplicitReturns": true,
"noFallthroughCasesInSwitch": true,
"exactOptionalPropertyTypes": true,
"removeComments": false,
"newLine": "lf",
"outDir": "../src/evalarc/assets",
"rootDir": "."
},
"files": ["report_enhance.ts"]
}
Loading
Loading