From 1d10d3c28d2ed23ecb82a65a46737b9141a1fa20 Mon Sep 17 00:00:00 2001 From: tekstrand Date: Tue, 25 Aug 2026 20:03:52 -0500 Subject: [PATCH] Added engine-level tests that push real signal through the decoder and the modulator. The existing js8core-lib tests cover lifecycle, audio submission and TX timing, but none of them proved the engine can actually recover a message, so a decoding regression would have passed the whole suite. Two new tests close that, on a small harness: TestEngine runs one engine per transmit or decode, shifts the clock for the job, and collects what the engine reports; Signal.kt holds the WAV and PCM helpers. That keeps each test to the thing it is testing, and the next engine test does not copy the decode path a third time. JS8EngineReferenceDecodeTest feeds the desktop project's own recordings into the decoder, which takes the microphone and the radio out of the picture entirely: it aligns the ring by shifting the engine clock to the top of a minute, submits the file, and reports what came back. The files are not copied into the tree; the androidTest source set packages media/tests as assets, so the recordings live in one place. It covers every submode A recording there. It is a smoke test rather than desktop parity, because the engine decodes a 13.6 second window at a fixed depth where the desktop CLI reads the whole file at the depth in the name. On the emulator the set gives 25 or 26 decodes across 6 of the 7 files against the desktop's 31. Every file but one gives the same count every run; A_2_5 has one marginal decode that comes and goes with how the ring lands, because the alignment is set from the Kotlin clock a few milliseconds before the native submit reads its own. The bounds sit under that rather than on it. Each decode's DT is logged, which is the ground truth for judging whether an alignment change actually helped. The clip goes in a buffer at a time rather than in one write. isDecodeReady works out which cycle the decode window belongs to from where a write ends, so a single write spanning the whole period puts the window in the next cycle, where nothing ever fills it and the pass never finishes. Feeding 4096 samples at a time, about what a device delivers, snaps the window while the write position is still in the first cycle. JS8EngineLoopbackTest goes the other way. It transmits, captures the waveform off the TX tap and decodes it, with no speaker or microphone anywhere in the path, so a failure is the transmitted audio itself. It asks for the transmission from the middle of a period, which is the case Modulator::start has to defer to the next slot boundary on its own, so it also guards that alignment: reverting that and running this test fails on a 0.14 second capture instead of a whole frame. Getting it to pass exposed something worth writing down. The tap sits after the output resampler, so it runs at whatever rate the audio device negotiated rather than the engine's 12000, which is 11520 on the emulator. Feeding those samples to a 12000 Hz decoder stretches everything by four percent and moves a 1500 Hz signal to 1562.5, which decodes as nothing at all. The harness resamples using the rate the tap reports. The Android build notes had steps for every kind of build and nothing about tests, so android/README.md gains a section for running the unit and instrumented suites, with the single-class form. Verified on the emulator: the eight instrumented tests pass, and the reference set was run nine times across two emulator boots to find the range above. The README commands were run as written. --- android/README.md | 24 +++ android/js8core-lib/build.gradle.kts | 5 + .../com/js8call/core/JS8EngineLoopbackTest.kt | 43 ++++++ .../core/JS8EngineReferenceDecodeTest.kt | 54 +++++++ .../java/com/js8call/core/Signal.kt | 81 ++++++++++ .../java/com/js8call/core/TestEngine.kt | 139 ++++++++++++++++++ 6 files changed, 346 insertions(+) create mode 100644 android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineLoopbackTest.kt create mode 100644 android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineReferenceDecodeTest.kt create mode 100644 android/js8core-lib/src/androidTest/java/com/js8call/core/Signal.kt create mode 100644 android/js8core-lib/src/androidTest/java/com/js8call/core/TestEngine.kt diff --git a/android/README.md b/android/README.md index b6ef1909..8e7a8935 100644 --- a/android/README.md +++ b/android/README.md @@ -40,6 +40,30 @@ JAVA_HOME=/opt/homebrew/opt/openjdk ANDROID_HOME=~/Library/Android/sdk ./gradlew Output: `android/app/build/outputs/apk/debug/` +## Run the tests + +Unit tests run on the host. From the repo android folder: + +```bash +JAVA_HOME=/opt/homebrew/opt/openjdk ANDROID_HOME=~/Library/Android/sdk ./gradlew :app:testDebugUnitTest +``` + +Report: `android/app/build/reports/tests/testDebugUnitTest/` + +The engine tests are instrumented and need a device or emulator attached: + +```bash +JAVA_HOME=/opt/homebrew/opt/openjdk ANDROID_HOME=~/Library/Android/sdk ./gradlew :js8core-lib:connectedDebugAndroidTest +``` + +Report: `android/js8core-lib/build/reports/androidTests/connected/` + +To run one class: + +```bash +JAVA_HOME=/opt/homebrew/opt/openjdk ANDROID_HOME=~/Library/Android/sdk ./gradlew :js8core-lib:connectedDebugAndroidTest -Pandroid.testInstrumentationRunnerArguments.class=com.js8call.core.JS8EngineLoopbackTest +``` + ## Build Release APK (signed) Create a keystore once: diff --git a/android/js8core-lib/build.gradle.kts b/android/js8core-lib/build.gradle.kts index aa1ceace..f4f85da5 100644 --- a/android/js8core-lib/build.gradle.kts +++ b/android/js8core-lib/build.gradle.kts @@ -112,6 +112,11 @@ android { "../../adapters/android/jni/java" ) } + getByName("androidTest") { + // The desktop project's reference recordings, packaged into the + // test APK rather than copied into the tree. + assets.srcDirs("../../media/tests") + } } } diff --git a/android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineLoopbackTest.kt b/android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineLoopbackTest.kt new file mode 100644 index 00000000..e9c3bae4 --- /dev/null +++ b/android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineLoopbackTest.kt @@ -0,0 +1,43 @@ +package com.js8call.core + +import android.util.Log +import androidx.test.ext.junit.runners.AndroidJUnit4 +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith + +/** + * Transmits through the modulator, captures the waveform off the TX tap and + * decodes it. No speaker or microphone in the path, so a failure here is the + * transmitted audio itself rather than the device. + * + * The transmission is asked for from the middle of a period, which is the case + * the modulator has to defer to the next slot boundary on its own. Starting + * partway through a frame sends a partial waveform, and a partial waveform does + * not decode, so this fails if that alignment is lost. + */ +@RunWith(AndroidJUnit4::class) +class JS8EngineLoopbackTest { + + @Test + fun ownTransmissionDecodes() { + val tx = TestEngine().use { it.transmitFromMidPeriod("CQ CQ CQ") } + + // A whole frame is 12.6 s. Short means the modulator joined one in + // progress instead of waiting for the boundary. + assertTrue("captured ${tx.seconds}s, expected a whole frame", tx.seconds > 10) + + // Submode A starts 500 ms into its 15 s period. The noise gives the + // decoder a floor to measure against instead of digital silence. + val period = tx.placedInPeriod(periodMs = 15_000, startDelayMs = 500) + .withNoise(rms = 300.0, seed = 1) + + val decoded = TestEngine().use { it.decode(period) } + Log.i(TAG, "decoded ${decoded.size}: $decoded") + assertTrue("own transmission did not decode", decoded.any { "CQ" in it.text }) + } + + private companion object { + const val TAG = "LoopbackTest" + } +} diff --git a/android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineReferenceDecodeTest.kt b/android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineReferenceDecodeTest.kt new file mode 100644 index 00000000..680f2bb7 --- /dev/null +++ b/android/js8core-lib/src/androidTest/java/com/js8call/core/JS8EngineReferenceDecodeTest.kt @@ -0,0 +1,54 @@ +package com.js8call.core + +import android.util.Log +import androidx.test.ext.junit.runners.AndroidJUnit4 +import org.junit.Assert.assertTrue +import org.junit.Test +import org.junit.runner.RunWith + +/** + * Feeds the desktop project's reference recordings straight into the engine, + * so a failure is the decoder rather than the microphone or the radio. The + * files are named MODE_DEPTH_EXPECTEDDECODES.wav and come from media/tests, + * packaged as test assets by the androidTest source set. + * + * This is a smoke test, not desktop parity: the engine decodes a 13.6 second + * window at a fixed depth, where the desktop CLI reads the whole file at the + * depth in the name, so the counts run under them. The counts are logged for + * comparison. On the emulator the set gives 25 or 26 decodes across 6 of the + * 7 files against the desktop's 31; one marginal decode in A_2_5 comes and + * goes with how the ring lands, which is why the bounds below sit under that + * rather than on it. + */ +@RunWith(AndroidJUnit4::class) +class JS8EngineReferenceDecodeTest { + + @Test + fun decodesReferenceRecordings() { + val results = REFERENCES.map { file -> + val decoded = TestEngine().use { it.decode(readWav(file)) } + Log.i(TAG, "$file: ${decoded.size} $decoded") + decoded + } + + val total = results.sumOf { it.size } + val files = results.count { it.isNotEmpty() } + Log.i(TAG, "decoded $total messages across $files of ${results.size} files") + assertTrue("only $total decodes from the reference set", total >= MIN_DECODES) + assertTrue("only $files files produced anything", files >= MIN_FILES) + } + + private companion object { + const val TAG = "RefDecodeTest" + + // Every submode A recording in media/tests. The engine decodes at a + // fixed depth, so files that differ only in that digit come back the same. + val REFERENCES = listOf( + "A_1_4.wav", "A_2_1.wav", "A_2_3.wav", "A_2_5.wav", + "A_2_6.wav", "A_2_9.wav", "A_3_3.wav", + ) + + const val MIN_DECODES = 20 + const val MIN_FILES = 5 + } +} diff --git a/android/js8core-lib/src/androidTest/java/com/js8call/core/Signal.kt b/android/js8core-lib/src/androidTest/java/com/js8call/core/Signal.kt new file mode 100644 index 00000000..1add6ba3 --- /dev/null +++ b/android/js8core-lib/src/androidTest/java/com/js8call/core/Signal.kt @@ -0,0 +1,81 @@ +package com.js8call.core + +import androidx.test.platform.app.InstrumentationRegistry +import java.nio.ByteBuffer +import java.nio.ByteOrder +import java.util.Random + +/** The engine's own rate. Everything here is 16-bit mono PCM at this rate. */ +internal const val SAMPLE_RATE = 12_000 + +internal val ShortArray.seconds: Float + get() = size / SAMPLE_RATE.toFloat() + +/** Drops the silence the TX tap delivers while the modulator waits for its slot. */ +internal fun ShortArray.trimLeadingSilence(): ShortArray { + val first = indexOfFirst { it.toInt() != 0 }.coerceAtLeast(0) + return copyOfRange(first, size) +} + +/** Lays the signal [startDelayMs] into an otherwise silent period of [periodMs]. */ +internal fun ShortArray.placedInPeriod(periodMs: Int, startDelayMs: Int): ShortArray { + val period = ShortArray(SAMPLE_RATE * periodMs / 1000) + val at = SAMPLE_RATE * startDelayMs / 1000 + copyInto(period, at, 0, minOf(size, period.size - at)) + return period +} + +/** Adds Gaussian noise for the decoder to measure against. Seeded, so runs repeat. */ +internal fun ShortArray.withNoise(rms: Double, seed: Long): ShortArray { + val rng = Random(seed) + return ShortArray(size) { i -> + (this[i] + rng.nextGaussian() * rms).toInt() + .coerceIn(Short.MIN_VALUE.toInt(), Short.MAX_VALUE.toInt()) + .toShort() + } +} + +/** Linear interpolation: enough to bring the tap's rate back to the engine's. */ +internal fun ShortArray.resampled(fromHz: Int, toHz: Int): ShortArray { + val out = ShortArray((size.toLong() * toHz / fromHz).toInt()) + for (i in out.indices) { + val pos = i.toDouble() * fromHz / toHz + val j = pos.toInt() + val a = this[j].toDouble() + val b = if (j + 1 < size) this[j + 1].toDouble() else a + out[i] = (a + (b - a) * (pos - j)).toInt().toShort() + } + return out +} + +internal fun List.concatenated(): ShortArray { + val out = ShortArray(sumOf { it.size }) + var at = 0 + for (part in this) { + part.copyInto(out, at) + at += part.size + } + return out +} + +/** A recording from the test assets. */ +internal fun readWav(name: String): ShortArray { + val bytes = InstrumentationRegistry.getInstrumentation().context.assets + .open(name).use { it.readBytes() } + val buf = ByteBuffer.wrap(bytes).order(ByteOrder.LITTLE_ENDIAN) + + // Walk the RIFF chunks; the recordings carry more than the 44 byte header. + var pos = 12 + while (pos + 8 <= bytes.size) { + val id = String(bytes, pos, 4, Charsets.US_ASCII) + val size = buf.getInt(pos + 4) + if (id == "data") { + val out = ShortArray(size / 2) + buf.position(pos + 8) + buf.asShortBuffer().get(out) + return out + } + pos += 8 + size + (size and 1) + } + throw IllegalStateException("no data chunk in $name") +} diff --git a/android/js8core-lib/src/androidTest/java/com/js8call/core/TestEngine.kt b/android/js8core-lib/src/androidTest/java/com/js8call/core/TestEngine.kt new file mode 100644 index 00000000..c0892e27 --- /dev/null +++ b/android/js8core-lib/src/androidTest/java/com/js8call/core/TestEngine.kt @@ -0,0 +1,139 @@ +package com.js8call.core + +import android.util.Log +import org.junit.Assert.assertTrue +import java.util.concurrent.CopyOnWriteArrayList + +/** One decode as the engine reported it. */ +internal data class Decode(val text: String, val snr: Int, val dt: Float, val freq: Float) { + override fun toString() = String.format("'%s' dt=%+.2f f=%.0f snr=%d", text, dt, freq, snr) +} + +/** + * A running engine for one transmit or one decode, closed by [use]. Submode A + * at the engine's own rate, TX tap on. Each job shifts the clock first, + * because the ring phase and the modulator both follow the wall clock. + */ +internal class TestEngine : AutoCloseable { + private val decodes = CopyOnWriteArrayList() + private val tapped = ArrayList() + private var tapRate = SAMPLE_RATE + @Volatile private var finished = false + + private val engine = JS8Engine.create( + sampleRateHz = SAMPLE_RATE, + submodes = SUBMODE_A, + enableTxAudioTap = true, + callbackHandler = object : JS8Engine.CallbackHandler { + override fun onDecoded( + utc: Int, snr: Int, dt: Float, freq: Float, text: String, + type: Int, quality: Float, mode: Int, driftMs: Int + ) { + decodes.add(Decode(text, snr, dt, freq)) + } + + override fun onDecodeFinished(count: Int) { + finished = true + } + + override fun onTxAudio(samples: ShortArray, sampleRateHz: Int) { + synchronized(tapped) { + tapRate = sampleRateHz + tapped.add(samples) + } + } + + override fun onError(message: String) { + Log.w(TAG, "engine error: $message") + } + + override fun onSpectrum( + bins: FloatArray, binHz: Float, powerDb: Float, peakDb: Float + ) = Unit + override fun onDecodeStarted(submodes: Int) = Unit + override fun onLog(level: Int, message: String) = Unit + } + ) + + init { + assertTrue("engine did not start", engine.start()) + } + + override fun close() = engine.close() + + /** Feeds the whole clip and waits for the decode pass over it to finish. */ + fun decode(samples: ShortArray): List { + // Shift the clock to the top of a minute; the first submit re-aligns + // the ring to it before writing. + val now = System.currentTimeMillis() + engine.setTimeDriftMs((MINUTE_MS - now % MINUTE_MS) % MINUTE_MS) + + // A buffer at a time, as a device would deliver it. The scheduler + // places the decode window from where a write ends, so one write + // spanning the whole period would put the window in the next one. + for (start in samples.indices step CHUNK) { + engine.submitAudio(samples.copyOfRange(start, minOf(start + CHUNK, samples.size))) + } + + assertTrue("decode pass never finished", waitUntil(DECODE_TIMEOUT_MS) { finished }) + Thread.sleep(SETTLE_MS) // decodes from that pass still arriving + return decodes.toList() + } + + /** + * Asks for [text] with the clock 13 s into a period, so the modulator has + * to defer to the next boundary on its own, and returns what came off the + * TX tap: at the engine rate, leading silence trimmed. + */ + fun transmitFromMidPeriod(text: String): ShortArray { + engine.setTransmitReady(true) + val offset = System.currentTimeMillis() % PERIOD_MS + engine.setTimeDriftMs((ASK_AT_MS - offset + PERIOD_MS) % PERIOD_MS) + val accepted = engine.transmitMessage( + text = text, + myCall = "N5EKS", + myGrid = "EM12", + submode = 0, + audioFrequencyHz = 1500.0, + txDelaySec = 0.0 + ) + assertTrue("transmit refused", accepted) + + val transmitting = { engine.isTransmittingAudio() } + assertTrue("modulator never started", waitUntil(TX_TIMEOUT_MS, transmitting)) + assertTrue("modulator never stopped", waitUntil(TX_TIMEOUT_MS) { !transmitting() }) + Thread.sleep(DRAIN_MS) // the last tap buffers are still in flight + engine.stopTransmit() + + // The tap sits after the output resampler, so it runs at whatever rate + // the audio device negotiated, not the engine rate the decoder wants. + val (raw, rate) = synchronized(tapped) { tapped.concatenated() to tapRate } + val tx = (if (rate == SAMPLE_RATE) raw else raw.resampled(rate, SAMPLE_RATE)) + .trimLeadingSilence() + Log.i(TAG, "tap at $rate Hz, ${raw.size} samples; ${tx.seconds}s of signal") + return tx + } + + private fun waitUntil(timeoutMs: Long, condition: () -> Boolean): Boolean { + val deadline = System.currentTimeMillis() + timeoutMs + while (!condition()) { + if (System.currentTimeMillis() >= deadline) return false + Thread.sleep(POLL_MS) + } + return true + } + + private companion object { + const val TAG = "TestEngine" + const val SUBMODE_A = 0x1 + const val PERIOD_MS = 15_000L + const val ASK_AT_MS = 13_000L // inside the frame, late enough to keep the wait short + const val MINUTE_MS = 60_000L + const val CHUNK = 4_096 + const val POLL_MS = 50L + const val SETTLE_MS = 500L + const val DRAIN_MS = 1_500L + const val TX_TIMEOUT_MS = 45_000L + const val DECODE_TIMEOUT_MS = 30_000L + } +}