benchmark-evaluation

When Better Means Less: Quantifying What Benchmarks Miss Between Model Generations. 2,310 controlled comparisons show GPT-5 series lost 6.7x creativity and gained 4.4x false refusals vs chatgpt-4o-latest — invisible to standard benchmarks.

Measure how much of a machine-learning benchmark score survives deployment. Grouped bootstrap, frozen thresholds, gap attribution.

Research Knowledge Acquisition Engine — 5 autonomous campaigns (literature-survey, patent-mining, benchmark-archaeology, meta-analysis, baseline-establishment) with quantitative budget enforcement

Code, data, and results for "The Correlation Mirage: Benchmark Dependence Collapses for Top-Performing LLMs" (EMNLP 2026). Copula-based tail dependence analysis of LLM benchmark suites.

Claim-relative identified sets across benchmark evaluation specifications

llmverify is a lightweight Python tool for externally auditing LLM APIs and verifying whether custom providers and resellers truly serve the frontier models they claim to run, using layered probes and benchmark-based statistics.

Does a benchmark overstate operational skill when the distribution shift is ordinary and physical? Houston ground-level ozone exceedance forecasting — the control domain in a multi-domain study of benchmark-vs-operational ML performance.

How much of a phishing benchmark's score is detection skill, and how much is an artifact of how the dataset was built? A zero-parameter regex scores TSS 0.99 on PhiUSIIL; the trained model scores 0.000 on live data.

Accepted to Findings of EMNLP 2026. Artifact for attributing function-calling gains to interface alignment versus procedural transfer.

Research record and qualification toolkit for the NeuroTopo architecture-candidate line: pre-registered protocols, reproducible runs, negative results

Key-free replication artifact for mutation testing of task-scoped state oracles in software-agent benchmarks

How Hard is Microservice Anomaly Detection?

SPIRAL: structured supervision harvesting and self-refining inference for surgical video understanding. ECCV 2026 MedVidU (Oral).

When Verified Financial Labels Fail: source-traced audits of released financial labels, rounding errors and paired grading consequences.

Audit of BixBench, FutureHouse's bioinformatics-agent benchmark: replicated runs, replicated grading, re-derived answer keys — and what a single pooled score hides.

Replication package for: The Score Is Not the Model - four protocol failure modes in Open Graph Benchmark reporting