Skip to content

Characterize static evaluation semantics - #365

Merged
uriahf merged 1 commit into
mainfrom
semantics/stage-1a-characterization
Aug 21, 2026
Merged

Characterize static evaluation semantics#365
uriahf merged 1 commit into
mainfrom
semantics/stage-1a-characterization

Conversation

@uriahf

@uriahf uriahf commented Aug 21, 2026

Copy link
Copy Markdown
Owner

Goal

Implement Stage 1A of the cross-language semantic audit as characterization tests only.

Coverage

Adds focused tests for:

  • one model × one population
  • multiple models × the same population
  • the same model/prediction rule across populations with different prevalence
  • distinct populations with equal prevalence
  • explicitly paired model-population inputs
  • calibration grouping and the global identity line
  • performance-table input semantics
  • ROC, precision-recall, gains, lift, decision, and interventions-avoided grouping/reference ownership
  • current model trace labels and color behavior

The tests deliberately preserve Python's current generic reference_group representation while distinguishing shared-population model cases from multi-population reference ownership.

Scope

Test-only. No statistical calculations, public APIs, production behavior, plotting code, package metadata, generated documentation, or rtichoke_viz are changed.

Any surprising current behavior is characterized rather than fixed in this PR.

Paired PR

R: uriahf/rtichoke#163

@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-08-21 10:52 UTC

@uriahf
uriahf marked this pull request as ready for review August 21, 2026 10:21
@uriahf
uriahf merged commit de8146e into main Aug 21, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant