Engine tests - #84
Merged
Merged
Conversation
…d the modulator. The existing js8core-lib tests cover lifecycle, audio submission and TX timing, but none of them proved the engine can actually recover a message, so a decoding regression would have passed the whole suite. Two new tests close that, on a small harness: TestEngine runs one engine per transmit or decode, shifts the clock for the job, and collects what the engine reports; Signal.kt holds the WAV and PCM helpers. That keeps each test to the thing it is testing, and the next engine test does not copy the decode path a third time. JS8EngineReferenceDecodeTest feeds the desktop project's own recordings into the decoder, which takes the microphone and the radio out of the picture entirely: it aligns the ring by shifting the engine clock to the top of a minute, submits the file, and reports what came back. The files are not copied into the tree; the androidTest source set packages media/tests as assets, so the recordings live in one place. It covers every submode A recording there. It is a smoke test rather than desktop parity, because the engine decodes a 13.6 second window at a fixed depth where the desktop CLI reads the whole file at the depth in the name. On the emulator the set gives 25 or 26 decodes across 6 of the 7 files against the desktop's 31. Every file but one gives the same count every run; A_2_5 has one marginal decode that comes and goes with how the ring lands, because the alignment is set from the Kotlin clock a few milliseconds before the native submit reads its own. The bounds sit under that rather than on it. Each decode's DT is logged, which is the ground truth for judging whether an alignment change actually helped. The clip goes in a buffer at a time rather than in one write. isDecodeReady works out which cycle the decode window belongs to from where a write ends, so a single write spanning the whole period puts the window in the next cycle, where nothing ever fills it and the pass never finishes. Feeding 4096 samples at a time, about what a device delivers, snaps the window while the write position is still in the first cycle. JS8EngineLoopbackTest goes the other way. It transmits, captures the waveform off the TX tap and decodes it, with no speaker or microphone anywhere in the path, so a failure is the transmitted audio itself. It asks for the transmission from the middle of a period, which is the case Modulator::start has to defer to the next slot boundary on its own, so it also guards that alignment: reverting that and running this test fails on a 0.14 second capture instead of a whole frame. Getting it to pass exposed something worth writing down. The tap sits after the output resampler, so it runs at whatever rate the audio device negotiated rather than the engine's 12000, which is 11520 on the emulator. Feeding those samples to a 12000 Hz decoder stretches everything by four percent and moves a 1500 Hz signal to 1562.5, which decodes as nothing at all. The harness resamples using the rate the tap reports. The Android build notes had steps for every kind of build and nothing about tests, so android/README.md gains a section for running the unit and instrumented suites, with the single-class form. Verified on the emulator: the eight instrumented tests pass, and the reference set was run nine times across two emulator boots to find the range above. The README commands were run as written.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
JS8EngineReferenceDecodeTest, which pushes the desktop project's reference recordings through the engine and counts what comes backJS8EngineLoopbackTest, which transmits a message, captures the modulator's own output, adds noise and decodes it againTestEngine, a small harness that runs one engine per transmit or decode and collects what it reports, andSignal.ktfor the WAV and PCM helpers.androidTestsource set atmedia/testsso the recordings are packaged into the test APK instead of copied into the treeandroid/README.md, which had build steps and nothing about testsWhy
Tests for the engine are nice while I develop to make sure I don't break stuff, and I got tired of running them manually between devices.
Test
./gradlew :js8core-lib:connectedDebugAndroidTestwith a device or emulator attached. Loopback takes about 18 seconds and the reference set about 7.