Skip to content

cli benchmark path tests - #288

Merged
yakew7 merged 2 commits into
yakew7:mainfrom
ahmdkaml:test/test-cli-py-benchmark-branch
Aug 21, 2026
Merged

cli benchmark path tests#288
yakew7 merged 2 commits into
yakew7:mainfrom
ahmdkaml:test/test-cli-py-benchmark-branch

Conversation

@ahmdkaml

Copy link
Copy Markdown
Collaborator

Summary of Changes

Added comprehensive test coverage for faircode/cli.py's benchmark subcommand (lines 245–283) in tests/test_cli.py[cite: 1]:

  • Missing Dependencies Handling: Verified that importing without the benchmark extra catches ImportError and outputs the pip install faircode[benchmark] install guide to stderr[cite: 1].
  • Paper-Run Drift Warning: Verified that non-default --n-resamples / --n-permutations trigger the reproducibility drift warning on stderr[cite: 1].
  • Missing Manifest Error Path: Confirmed the exit code and error message when no audit.yaml files are found under the search root[cite: 1].
  • CLI Success Execution: Validated the complete CLI execution against German Credit Lending/audit.yaml, confirming exit code 0, summary output on stderr, and artifact generation (results_fairness.csv, results_performance.csv, summary.csv)[cite: 1, 2].

@ahmdkaml
ahmdkaml requested a review from yakew7 as a code owner August 21, 2026 04:20
@vercel

vercel Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

@ahmdkaml is attempting to deploy a commit to the yashkewlani2020-gmailcom's projects Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown

@yakew7 - new PR touching a path you own, please review.

@yakew7
yakew7 merged commit 1ee2943 into yakew7:main Aug 21, 2026
16 of 17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants