Skip to content

feat: add labeled benchmark corpus and quality metrics - #49

Merged
CoreyLeath-code merged 12 commits into
mainfrom
codex/labeled-benchmark-metrics
Nov 13, 2026
Merged

CoreyLeath-code merged 12 commits into
mainfrom
codex/labeled-benchmark-metrics

Conversation

@CoreyLeath-code

Copy link
Copy Markdown
Owner

Summary

  • add a versioned synthetic JSONL corpus and manifest with an explicit anomaly-label contract
  • add deterministic precision, recall, F1, incident recall, and alert-burden evaluation
  • expose logsight benchmark with optional deployment-owned precision, recall, and alert-volume gates
  • document corpus governance and mark the final production-roadmap item complete

Data and safety boundary

The committed corpus is synthetic only. It contains no customer data, credentials, or production logs. Gate values are supplied by the release/deployment process rather than hard-coded as universal production thresholds.

Validation

GitHub Actions will validate formatting, linting, strict typing, tests/coverage, benchmark reproducibility, package/container construction, security controls, SBOM, Helm, and CodeQL.

@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-13T22:24:07.793372Z
We couldn’t complete this review. Comment @codex review to try again.
7a607a9 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@coderabbitai

coderabbitai Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 28 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 82b95ee6-99fe-4938-9a4d-0bf886a23e3b
Commits

Reviewing files that changed from the base of the PR and between 86af0c3 and f53d058.

Files selected for processing: 9

  • README.md
  • benchmarks/corpus/v1/events.jsonl
  • benchmarks/corpus/v1/manifest.json
  • docs/BENCHMARKING.md
  • docs/LABELED_BENCHMARKS.md
  • logsight/benchmarking.py
  • logsight/cli.py
  • tests/test_benchmarking.py
  • tests/test_cli.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@CoreyLeath-code
CoreyLeath-code merged commit f34069b into main Nov 13, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant