Add a golden/regression test suite and CI#18
Conversation
Adds the first automated tests for the PLAAC CLI, which previously had
none. PLAAC's scoring is deterministic (Viterbi parse over fixed
frequency tables; no randomness, timestamps, or host paths in the
output), so its output can be guarded byte-for-byte.
cli/tests/run_tests.sh (plain bash + java, no test-framework dependency):
1. golden -- full output of the bundled four_classic_prions.fasta at
-a 1 -c 60 -d is compared byte-for-byte against a
committed expected file. Locks the scored output and the
built-in frequency/column tables (from which the web
HAML is generated) against unintended changes to the
~5k-line plaac.java.
2. recovery -- Sup35p/Ure2p/Rnq1p/Mot3p each get a called PrLD
(PRDlen > 0); a semantic check that survives a reviewed
regeneration of the golden file.
The harness pins the JVM locale so number formatting is reproducible on
CI hosts with a comma-decimal default locale. tests/regenerate_golden.sh
regenerates the golden file for reviewed, intentional numeric changes.
.github/workflows/ci.yml builds plaac.jar and runs the suite on push/PR,
making 'still builds and still produces correct scores' automatic.
|
Thanks for maintaining PLAAC! After opening this I saw that the I have opened #19 against |
Hi maintainers, I use PLAAC in my routine workflows and wanted to add automated tests for the PLAAC CLI, plus CI to run them.
The scoring logic in
cli/src/plaac.javacurrently has no automated test guarding it, so there is no safety net against an edit unintentionally changing results. PLAAC’s output is deterministic, which makes byte-for-byte regression testing straightforward.What this adds (pure addition — no changes to
plaac.java)cli/tests/run_tests.sh— plainbash+java, no test-framework dependency. Two checks:cli/example/four_classic_prions.fastaat-a 1 -c 60 -dand compares the full output byte-for-byte against a committed expected file. This locks the scored output and the built-in frequency/column tables (from which the web_plaac_headers.hamlis generated).PRDlen > 0), a semantic check that survives a reviewed, intentional regeneration of the golden file.cli/tests/expected/four_classic_prions.a1.c60.txt— the golden file.cli/tests/regenerate_golden.sh— regenerates the golden for reviewed numeric changes.cli/tests/README.md— docs..github/workflows/ci.yml— buildsplaac.jarand runs the suite on push/PR (JDK 17), and uploads the actual output as an artifact.Reproducibility
PLAAC scores use
Math.log/Math.exp(platform intrinsics), so I verified the output is byte-identical across macOS/arm64 and Linux/x64 (Temurin 17) before committing; the harness also pins the JVM locale so number formatting is host-independent. CI is green.Notes
cli/tests/and.github/is touched.