Temporal evaluation and representative subset selection

[henriquejsb]

This pull request introduces two major new commands to the ethibench CLI: temporal and subset. The temporal command generates cumulative evaluation plots over elapsed time, while the subset command finds the most representative subset of targets under a cost constraint. The documentation and CLI have been updated to reflect these additions, and new modules have been added to support the functionality. Additionally, an experiment-specific script for temporal evaluation has been removed in favor of the new, general-purpose implementation.

New CLI commands and features:

• Added the ethibench temporal command, which generates a 4×N subplot figure showing cumulative true positives, false positives, severity score, and CWE coverage over time for each run and target. This command requires findings with timestamps and that ethibench evaluate has already been executed. [1] [2]
• Added the ethibench subset command, which selects a subset of targets whose F1 scores best correlate with the full benchmark, subject to a cost ratio constraint. This helps users create a reduced evaluation suite that remains representative while reducing cost. [1] [2]

Documentation and codebase updates:

• Updated the README.md to document the new temporal and subset commands, including usage, options, and behavior. Also updated the module tree to include the new temporal.py and subset.py files. [1] [2] [3]
• Updated the CLI entry points comment in README.md to mention the new commands.

Code cleanup:

• Removed the experiment-specific script experiments/claude-code-sonnet/temporal_evaluation.py, as its functionality is now provided by the general-purpose ethibench temporal command.

[github-actions]

🎉 This PR is included in version 1.1.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀