Catalogue three published benchmarks, without shipping them as profiles

[emmanuelgjr]

Handles the "Convert" half of the watcher research triage: #47 LongPIBench, #55 GenIaC-SecBench, #70 TIER.

What this adds

evals/EXTERNAL_BENCHMARKS.md — a catalogue of published evaluation corpora that test the risks this crosswalk maps, with a row per benchmark giving what it measures, its scale as published, a DRAFT OWASP entry and its arXiv link.

No Garak or PyRIT profile is added, per your instruction. The file says so in its own words: not runnable from here, not threshold-bearing (a pass mark for someone else's benchmark is theirs to set, so THRESHOLDS.md is untouched), and not a reproduction — every figure is the authors', transcribed from the abstracts, nothing re-run. evals/garak/, evals/pyrit/ and evals/laaf/ remain the only runnable profiles, so the existing threshold tests are unaffected.

Benchmark	Scale, as published	Entry (DRAFT)
LongPIBench	4 scenarios — peer review, resume screening, code review, email summary — synthetic + real-world datasets, thousands to tens of thousands of tokens	LLM01
GenIaC-SecBench	100 deployment scenarios, 12 model configurations from 4 vendors, 1,196 IaC artifacts, 3 scanners	LLM10
TIER	4 risk domains × 4 threat levels, 6-label behaviour scale, 2 LLM judges, 6 open-weight models	LLM01

The entry column is marked DRAFT — SME review required: reading a benchmark's scope onto an OWASP entry is a mapping judgment, so it is proposed, not asserted.

It records the rule, not just the result

The file states why the other nine items in that queue are absent: a paper earns a row only if it publishes a benchmark or evaluation corpus. Techniques, frameworks and vulnerability analyses are literature — and none of them belongs in data/incidents.json, which stays reserved for real-world deployment failures.

Verification

• Abstracts read in full from the arXiv API, not the truncated excerpts in the issues — that is where LongPIBench's fourth scenario (email summary) and the GenIaC figures come from.
• All three arXiv links return 200.
• node scripts/validate.js: 0 errors. npm run test:scripts: 85 passed, 0 failed.

Once this merges I'll comment and close all twelve research issues: these three pointing here, the other nine as "Theoretical/Reference — not a deployment failure".

🤖 Generated with Claude Code