Skip to content

test(kgemma): re-enable the Gemma 4 E2B tool-call smoke test — passes under engine 0.53.0 #399

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
michalharakal merged 2 commits into from
Sep 2, 2026
Merged

test(kgemma): re-enable the Gemma 4 E2B tool-call smoke test — passes under engine 0.53.0 #399

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dc08abd
test(kgemma): re-enable the Gemma 4 E2B tool-call smoke test — passes…
michalharakal Sep 2, 2026
dfd1c18
test(kgemma): drop the now-unused Ignore import
michalharakal Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions llm-runtime/kgemma/src/jvmTest/kotlin/sk/ainet/apps/kgemma/cli/Gemma4E2BToolCallSmokeTest.kt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,6 @@ import kotlin.io.path.exists
import kotlin.io.path.isDirectory
import kotlin.io.path.readText
import kotlin.random.Random
import kotlin.test.Ignore
import kotlin.test.Test
import kotlin.test.assertTrue
import kotlin.test.fail
Expand Down Expand Up @@ -118,9 +117,10 @@ class Gemma4E2BToolCallSmokeTest {
}

@Test
@Ignore // Gemma 4 E2B emits coherent English on the calculator prompt but no <|tool_call> markup.
// Format-grammar gap is upstream of the agent loop (suspected prefill/sampling divergence).
// Re-enable once the underlying fix lands; the assertions here are the regression guard.
// Was @Ignore'd ("emits coherent English but no <|tool_call> markup"). Under engine
// 0.53.0 the real E2B Q4_K_M checkpoint emits `<|tool_call>call:calculator{expression:...}`
// and this test passes (~80 s on an M-series host); it stays env-gated on
// GEMMA4_E2B_MODEL_PATH, so CI without the checkpoint still skips.
fun `real Gemma 4 E2B emits parseable tool_call against Gemma4ChatTemplate`() {
val modelPath = System.getenv("GEMMA4_E2B_MODEL_PATH")?.trim().orEmpty()
if (modelPath.isEmpty()) {
Expand Down