Skip to content

Package a reusable CI/CD regression-detection workflow template for downstream repos #355

Description

@himanshu231204

Problem

oaeval test and the cicd/ threshold-gate machinery already exist and work, but there's no packaged, drop-in GitHub Actions template showing another repository how to fail its own CI on an evaluation regression. Every adopter currently has to hand-roll this.

Goal

A developer can add regression-gated evaluation to their own repo's CI by copying one file, not by reverse-engineering cicd/plugin.py.

Suggested implementation

Publish either a reusable composite GitHub Action (.github/actions/oaeval-gate/) or a documented workflow snippet (docs/cicd-template.md) that runs oaeval test config.yaml -t <thresholds> and fails the job on threshold violation, with a worked example using the mock provider so it's runnable with zero secrets in a demo.

Files/components

  • .github/actions/oaeval-gate/ (proposed new composite action) or docs/cicd-template.md (proposed new doc)
  • docs/13_cicd_integration.md (existing — cross-link)

Acceptance criteria

  • A copy-pasteable template exists and is demonstrated working (e.g. in this repo's own CI or a linked example repo).

Difficulty

Advanced


Filed from a repository audit. Full report: https://claude.ai/code/artifact/134cec04-96c5-4d58-af5e-207def4ab524

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions