diff --git a/KernelRace/README.md b/KernelRace/README.md index 932d853..2d3f181 100644 --- a/KernelRace/README.md +++ b/KernelRace/README.md @@ -12,24 +12,46 @@ hand-written NEON matmul kernels behind a JNI bridge, with two `.so` tiers (`arm This example began as the Android-only **AndroidNeonLlmDemo** (still available as a frozen snapshot at the git tag [`2026_08_arm_android`](../../tree/2026_08_arm_android/AndroidNeonLlmDemo)) and was converted into a proper Kotlin Multiplatform sample: shared engine/view-model logic, a -Compose Multiplatform UI, and platform-specific runtime construction for Android, Desktop and -Wasm. iOS is deferred until the multiplatform structure is proven out. +Compose Multiplatform UI, and platform-specific runtime construction for Android, Desktop, Wasm +and iOS. + +On iOS, the same Llama runtime dispatches to SKaiNET's Apple `native-cinterop` packed-quant +kernels (see `shared/src/iosMain/.../Platform.ios.kt`) — the direct engine-level counterpart to +Android's NEON JNI kernels, though without the two-process split-screen race mechanic (Android +only, see below). + +> **Note**: this repo's `iosApp/` Xcode project was authored without access to Xcode/macOS, so +> it's structurally complete but unbuilt — the first real compile/run needs a Mac. If Xcode +> reports a project-format issue on first open, that's the thing to fix; the Kotlin side +> (`shared`/`composeApp`'s `iosArm64`/`iosSimulatorArm64` targets) is unaffected by it either way. +> Separately: `kllama`'s Apple native-kernel wiring was fixed in SKaiNET-transformers PR #315 +> (stale no-op stubs were leaving packed-quant matmul on the scalar floor on iOS/macOS) — until a +> SK-TR release ships that fix, KernelRace's iOS build runs correctly but without NEON +> acceleration. ## What it demonstrates - **~5-line integration**: `DecoderGgufWeightLoader` → `OptimizedLLMRuntime` → `generateUntilStop`, streaming tokens into Compose (see `shared/.../engine/LlmEngine.kt` and the per-platform `LlamaRuntimeBuilder.*.kt` actuals). +- **How cheap adding iOS actually was**: the entire iOS-specific surface is ~200 lines across + four files (`shared/src/iosMain/`, `composeApp/src/iosMain/`) — and the one that matters, + `LlamaRuntimeBuilder.ios.kt`, is 36 lines and nearly a line-for-line copy of the JVM actual + (same `DecoderGgufWeightLoader` call, `PosixPreadRandomAccessSource` instead of + `JvmRandomAccessSource`). No SKaiNET code changed to make this work — the engine's KMP targets + and Apple `native-cinterop` kernels were already there. That's the actual point of this sample: + proof that SKaiNET apps aren't Android-first with iOS bolted on — iOS is just another + `expect`/`actual` pair. - **NEON | SCALAR switch** (Android only): two chips re-pin the kernel registry (engine reloads on the next run) — same APK, same model, full-device A/B with a live tok/s counter. - **Split-screen race** (Android only): one button launches a second process with the scalar provider pinned and starts both generations simultaneously. ![Split-screen race: NEON at 44.7 tok/s vs scalar at 9.3 tok/s](docs/screenshots/split_race.png) -- **Cross-platform kernel tiers**: the same Kotlin `LlmEngine` runs on three different kernel - paths — Android's ARM NEON JNI kernels, Desktop's native-optimized file-based load, and Wasm's - in-memory FP32 fallback (browsers have no filesystem, so the model is bundled at build time - instead of downloaded). +- **Cross-platform kernel tiers**: the same Kotlin `LlmEngine` runs on four different kernel + paths — Android's ARM NEON JNI kernels, iOS's Apple `native-cinterop` kernels, Desktop's + native-optimized file-based load, and Wasm's in-memory FP32 fallback (browsers have no + filesystem, so the model is bundled at build time instead of downloaded). - **Model delivery**: Android downloads the GGUF from the Hugging Face Hub on first run (SKaiNET's Ktor fetcher, streamed to disk with progress) or uses a bundled asset if present; Desktop downloads to a local cache dir; Wasm bundles the model into the production build via @@ -76,12 +98,37 @@ That's a deliberately large asset for a web page — acceptable for a local dev maintainer-approved deploy, but worth knowing about before wiring this into CI or a public samples page. +The production build (as deployed to GitHub Pages) vendors the +[`coi-serviceworker`](https://github.com/gzuidhof/coi-serviceworker) shim — Compose's Wasm/Skiko +canvas needs cross-origin isolation (`SharedArrayBuffer`) for its multi-threaded renderer, and +GitHub Pages can't set the `Cross-Origin-Opener-Policy`/`Cross-Origin-Embedder-Policy` response +headers that provides. `wasmJsBrowserDevelopmentRun`'s own dev server sets them automatically, so +this only matters for the static production build. + +### iOS + +```sh +open iosApp/iosApp.xcodeproj +``` + +Run the `iosApp` scheme on a simulator or device from Xcode (⌘R). First build runs a "Compile +Kotlin Framework" script phase (`:composeApp:embedAndSignAppleFrameworkForXcode`) that builds and +embeds `shared`+`composeApp`'s Kotlin/Native framework before Swift compiles — no separate Gradle +step needed. Model delivery mirrors Desktop: downloads to the app's Caches directory on first run, +or bundle `SmolLM2-135M-Instruct-Q8_0.gguf` into the Xcode target for an offline build. + +No NEON-vs-scalar race UI here (Android-only mechanic) — but the Llama runtime still dispatches +through SKaiNET's Apple `native-cinterop` kernels rather than falling back to scalar, once the +`kllama` fix referenced above has shipped. + ## Requirements - Android: ARM64 device (minSdk 24, one APK covers armv8.0 through armv9), JDK 21+ - Desktop: JDK 21+ with the JDK Vector API (incubator) enabled — wired automatically by the Gradle build - Web: a Chromium-based browser for the wasm GC runtime +- iOS: Xcode 15+, a Mac (this repo's iOS support was authored and structurally verified without + either — see the note above) ## Architecture @@ -92,13 +139,16 @@ KernelRace/ │ ├── commonMain/ # LlmEngine, ChatViewModel, ModelResolver (pure, unit-tested) │ ├── androidMain/ # AndroidModelProvider, NEON-aware LlamaRuntimeBuilder actual │ ├── jvmMain/ # DesktopModelProvider, file-based LlamaRuntimeBuilder actual +│ ├── iosMain/ # IosModelProvider (Ktor/Darwin), pread-based LlamaRuntimeBuilder actual │ └── wasmJsMain/ # Bytes-only LlamaRuntimeBuilder actual (no filesystem) └── composeApp/ # Compose Multiplatform UI + platform entry points └── src/ ├── commonMain/ # App/ChatScreen (skainet-ui themed), kernelControls slot ├── androidMain/ # KernelRaceApp (kernel pinning), race UI, manifest ├── jvmMain/ # Desktop window entry point + ├── iosMain/ # MainViewController — entry point called from iosApp/ └── wasmJsMain/ # Browser entry point + bundled model resource +iosApp/ # Xcode project shell embedding composeApp's Kotlin/Native framework ``` The race mechanics (multi-process kernel pinning, `KernelRegistry`, the split-screen button) are diff --git a/KernelRace/THIRD_PARTY_LICENSES/MIT-coi-serviceworker.txt b/KernelRace/THIRD_PARTY_LICENSES/MIT-coi-serviceworker.txt new file mode 100644 index 0000000..4932a51 --- /dev/null +++ b/KernelRace/THIRD_PARTY_LICENSES/MIT-coi-serviceworker.txt @@ -0,0 +1,21 @@ +MIT License + +Copyright (c) 2021 Guido Zuidhof + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. diff --git a/KernelRace/THIRD_PARTY_LICENSES/NOTICE b/KernelRace/THIRD_PARTY_LICENSES/NOTICE index 19aba81..6341e6e 100644 --- a/KernelRace/THIRD_PARTY_LICENSES/NOTICE +++ b/KernelRace/THIRD_PARTY_LICENSES/NOTICE @@ -7,3 +7,9 @@ GGUF build: https://huggingface.co/unsloth/SmolLM2-135M-Instruct-GGUF GGUF file: SmolLM2-135M-Instruct-Q8_0.gguf (~145 MB) Attribution: HuggingFaceTB. See https://github.com/huggingface/smollm. + +This product bundles coi-serviceworker (composeApp/src/wasmJsMain/resources/ +coi-serviceworker.js), (c) 2021 Guido Zuidhof and contributors, distributed +under the MIT License (see MIT-coi-serviceworker.txt in this directory). + +Source: https://github.com/gzuidhof/coi-serviceworker diff --git a/KernelRace/composeApp/build.gradle.kts b/KernelRace/composeApp/build.gradle.kts index 10b1033..098633c 100644 --- a/KernelRace/composeApp/build.gradle.kts +++ b/KernelRace/composeApp/build.gradle.kts @@ -21,6 +21,13 @@ kotlin { jvm() + listOf(iosArm64(), iosSimulatorArm64()).forEach { target -> + target.binaries.framework { + baseName = "ComposeApp" + isStatic = true + } + } + @OptIn(ExperimentalWasmDsl::class) wasmJs { browser() diff --git a/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/ChatScreen.kt b/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/ChatScreen.kt index 0cdc132..da5dfb5 100644 --- a/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/ChatScreen.kt +++ b/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/ChatScreen.kt @@ -82,6 +82,8 @@ fun ChatScreen( Modifier.fillMaxSize().padding(padding) .padding(horizontal = 16.dp, vertical = 4.dp) ) { + PlatformChips(Modifier.padding(bottom = 6.dp)) + kernelControls(state.busy, viewModel) { prompt } Text( diff --git a/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/PlatformChips.kt b/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/PlatformChips.kt new file mode 100644 index 0000000..751267b --- /dev/null +++ b/KernelRace/composeApp/src/commonMain/kotlin/sk/ainet/samples/kernelrace/ui/PlatformChips.kt @@ -0,0 +1,34 @@ +package sk.ainet.samples.kernelrace.ui + +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.material3.ExperimentalMaterial3Api +import androidx.compose.material3.FilterChip +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.ui.Modifier +import androidx.compose.ui.unit.dp +import sk.ainet.samples.kernelrace.platform.SamplePlatform +import sk.ainet.samples.kernelrace.platform.currentSamplePlatform + +/** + * One chip per platform this sample runs on, with whichever one is currently running + * highlighted — a quick visual "this sample is genuinely multiplatform" signal, independent of + * [sk.ainet.samples.kernelrace.platform.kernelTierLabel] (which describes the kernel *inside* + * the current platform, not the platform list itself). Purely informational — selection is + * driven by [currentSamplePlatform], not by taps. + */ +@OptIn(ExperimentalMaterial3Api::class) +@Composable +fun PlatformChips(modifier: Modifier = Modifier) { + Row(modifier.fillMaxWidth(), horizontalArrangement = Arrangement.spacedBy(6.dp)) { + for (platform in SamplePlatform.entries) { + FilterChip( + selected = platform == currentSamplePlatform, + onClick = {}, + label = { Text(platform.label) }, + ) + } + } +} diff --git a/KernelRace/composeApp/src/iosMain/kotlin/sk/ainet/samples/kernelrace/MainViewController.kt b/KernelRace/composeApp/src/iosMain/kotlin/sk/ainet/samples/kernelrace/MainViewController.kt new file mode 100644 index 0000000..719c343 --- /dev/null +++ b/KernelRace/composeApp/src/iosMain/kotlin/sk/ainet/samples/kernelrace/MainViewController.kt @@ -0,0 +1,24 @@ +package sk.ainet.samples.kernelrace + +import androidx.compose.runtime.remember +import androidx.compose.ui.window.ComposeUIViewController +import platform.UIKit.UIViewController +import sk.ainet.context.DirectCpuExecutionContext +import sk.ainet.samples.kernelrace.engine.LlmEngine +import sk.ainet.samples.kernelrace.model.IosModelProvider +import sk.ainet.samples.kernelrace.model.ModelData +import sk.ainet.samples.kernelrace.vm.ChatViewModel + +/** Entry point called from iosApp/iOSApp.swift — same "resolve model, then load engine" shape + * as the JVM/Android entry points (see main.kt / MainActivity.kt). No kernelControls slot: + * the NEON-vs-scalar race is Android-only (see Platform.ios.kt's supportsKernelRace = false). */ +fun MainViewController(): UIViewController = ComposeUIViewController { + val viewModel = remember { + ChatViewModel(loadModel = { onProgress -> + val model = IosModelProvider().resolve(onProgress) as ModelData.FilePath + onProgress("Building runtime…") + LlmEngine.load(DirectCpuExecutionContext(), model) + }) + } + App(viewModel = viewModel, skainetVersion = SKAINET_VERSION) +} diff --git a/KernelRace/composeApp/src/wasmJsMain/resources/app.html b/KernelRace/composeApp/src/wasmJsMain/resources/app.html index 9aa46fc..cda8780 100644 --- a/KernelRace/composeApp/src/wasmJsMain/resources/app.html +++ b/KernelRace/composeApp/src/wasmJsMain/resources/app.html @@ -5,6 +5,13 @@ Kernel Race + + diff --git a/KernelRace/composeApp/src/wasmJsMain/resources/coi-serviceworker.js b/KernelRace/composeApp/src/wasmJsMain/resources/coi-serviceworker.js new file mode 100644 index 0000000..9901474 --- /dev/null +++ b/KernelRace/composeApp/src/wasmJsMain/resources/coi-serviceworker.js @@ -0,0 +1,146 @@ +/*! coi-serviceworker v0.1.7 - Guido Zuidhof and contributors, licensed under MIT */ +let coepCredentialless = false; +if (typeof window === 'undefined') { + self.addEventListener("install", () => self.skipWaiting()); + self.addEventListener("activate", (event) => event.waitUntil(self.clients.claim())); + + self.addEventListener("message", (ev) => { + if (!ev.data) { + return; + } else if (ev.data.type === "deregister") { + self.registration + .unregister() + .then(() => { + return self.clients.matchAll(); + }) + .then(clients => { + clients.forEach((client) => client.navigate(client.url)); + }); + } else if (ev.data.type === "coepCredentialless") { + coepCredentialless = ev.data.value; + } + }); + + self.addEventListener("fetch", function (event) { + const r = event.request; + if (r.cache === "only-if-cached" && r.mode !== "same-origin") { + return; + } + + const request = (coepCredentialless && r.mode === "no-cors") + ? new Request(r, { + credentials: "omit", + }) + : r; + event.respondWith( + fetch(request) + .then((response) => { + if (response.status === 0) { + return response; + } + + const newHeaders = new Headers(response.headers); + newHeaders.set("Cross-Origin-Embedder-Policy", + coepCredentialless ? "credentialless" : "require-corp" + ); + if (!coepCredentialless) { + newHeaders.set("Cross-Origin-Resource-Policy", "cross-origin"); + } + newHeaders.set("Cross-Origin-Opener-Policy", "same-origin"); + + return new Response(response.body, { + status: response.status, + statusText: response.statusText, + headers: newHeaders, + }); + }) + .catch((e) => console.error(e)) + ); + }); + +} else { + (() => { + const reloadedBySelf = window.sessionStorage.getItem("coiReloadedBySelf"); + window.sessionStorage.removeItem("coiReloadedBySelf"); + const coepDegrading = (reloadedBySelf == "coepdegrade"); + + // You can customize the behavior of this script through a global `coi` variable. + const coi = { + shouldRegister: () => !reloadedBySelf, + shouldDeregister: () => false, + coepCredentialless: () => true, + coepDegrade: () => true, + doReload: () => window.location.reload(), + quiet: false, + ...window.coi + }; + + const n = navigator; + const controlling = n.serviceWorker && n.serviceWorker.controller; + + // Record the failure if the page is served by serviceWorker. + if (controlling && !window.crossOriginIsolated) { + window.sessionStorage.setItem("coiCoepHasFailed", "true"); + } + const coepHasFailed = window.sessionStorage.getItem("coiCoepHasFailed"); + + if (controlling) { + // Reload only on the first failure. + const reloadToDegrade = coi.coepDegrade() && !( + coepDegrading || window.crossOriginIsolated + ); + n.serviceWorker.controller.postMessage({ + type: "coepCredentialless", + value: (reloadToDegrade || coepHasFailed && coi.coepDegrade()) + ? false + : coi.coepCredentialless(), + }); + if (reloadToDegrade) { + !coi.quiet && console.log("Reloading page to degrade COEP."); + window.sessionStorage.setItem("coiReloadedBySelf", "coepdegrade"); + coi.doReload("coepdegrade"); + } + + if (coi.shouldDeregister()) { + n.serviceWorker.controller.postMessage({ type: "deregister" }); + } + } + + // If we're already coi: do nothing. Perhaps it's due to this script doing its job, or COOP/COEP are + // already set from the origin server. Also if the browser has no notion of crossOriginIsolated, just give up here. + if (window.crossOriginIsolated !== false || !coi.shouldRegister()) return; + + if (!window.isSecureContext) { + !coi.quiet && console.log("COOP/COEP Service Worker not registered, a secure context is required."); + return; + } + + // In some environments (e.g. Firefox private mode) this won't be available + if (!n.serviceWorker) { + !coi.quiet && console.error("COOP/COEP Service Worker not registered, perhaps due to private mode."); + return; + } + + n.serviceWorker.register(window.document.currentScript.src).then( + (registration) => { + !coi.quiet && console.log("COOP/COEP Service Worker registered", registration.scope); + + registration.addEventListener("updatefound", () => { + !coi.quiet && console.log("Reloading page to make use of updated COOP/COEP Service Worker."); + window.sessionStorage.setItem("coiReloadedBySelf", "updatefound"); + coi.doReload(); + }); + + // If the registration is active, but it's not controlling the page + if (registration.active && !n.serviceWorker.controller) { + !coi.quiet && console.log("Reloading page to make use of COOP/COEP Service Worker."); + window.sessionStorage.setItem("coiReloadedBySelf", "notcontrolling"); + coi.doReload(); + } + }, + (err) => { + !coi.quiet && console.error("COOP/COEP Service Worker failed to register:", err); + } + ); + })(); +} diff --git a/KernelRace/gradle/libs.versions.toml b/KernelRace/gradle/libs.versions.toml index b1c39d3..b0b21fe 100644 --- a/KernelRace/gradle/libs.versions.toml +++ b/KernelRace/gradle/libs.versions.toml @@ -15,6 +15,10 @@ composeMultiplatform = "1.10.1" kotlin = "2.4.10" kotlinx-coroutines = "1.11.0" kotlinxIo = "0.9.0" +# Matches the engine repo's own pin (gradle/libs.versions.toml). Used directly by +# IosModelProvider: skainet-data-source is JVM-only (KtorRemoteDataSourceFetcher lives in +# its jvmMain), so the iOS download path talks to Ktor's Darwin engine on its own. +ktorClientCore = "3.5.2" # skainet-transformers has not released against the 0.40.x core line yet # (tops out at 0.39.1 on Maven Central), so this sample — like KllamaDemo — # stays one core version behind the transformers-free samples. @@ -33,11 +37,18 @@ kotlinx-coroutines-test = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-t kotlinx-coroutines-android = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-android", version.ref = "kotlinx-coroutines" } kotlinx-coroutinesSwing = { module = "org.jetbrains.kotlinx:kotlinx-coroutines-swing", version.ref = "kotlinx-coroutines" } kotlinx-io-core = { module = "org.jetbrains.kotlinx:kotlinx-io-core", version.ref = "kotlinxIo" } +ktor-client-core = { module = "io.ktor:ktor-client-core", version.ref = "ktorClientCore" } +ktor-client-darwin = { module = "io.ktor:ktor-client-darwin", version.ref = "ktorClientCore" } # SKaiNET core — BOM-governed skainet-bom = { module = "sk.ainet:skainet-bom", version.ref = "skainet" } skainet-lang-core = { module = "sk.ainet.core:skainet-lang-core" } skainet-backend-cpu = { module = "sk.ainet.core:skainet-backend-cpu" } +# Compile-time only: PanamaVectorKernelProvider implements KernelProvider (declared here), +# so resolving .isAvailable() in Platform.jvm.kt needs this on the classpath even though it's +# never referenced by name — skainet-backend-cpu depends on it as implementation-scoped, so +# it isn't reachable transitively. +skainet-backend-api = { module = "sk.ainet.core:skainet-backend-api" } # Hand-written ARM NEON kernels — Android/JVM JNI only, not wired into commonMain. skainet-backend-jni-cpu = { module = "sk.ainet.core:skainet-backend-jni-cpu" } skainet-data-source = { module = "sk.ainet.core:skainet-data-source" } diff --git a/KernelRace/iosApp/iosApp.xcodeproj/project.pbxproj b/KernelRace/iosApp/iosApp.xcodeproj/project.pbxproj new file mode 100644 index 0000000..8db11b4 --- /dev/null +++ b/KernelRace/iosApp/iosApp.xcodeproj/project.pbxproj @@ -0,0 +1,288 @@ +// !$*UTF8*$! +{ + archiveVersion = 1; + classes = { + }; + objectVersion = 56; + objects = { + +/* Begin PBXBuildFile section */ + 1A00000000000000000001 /* iOSApp.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1A00000000000000000011 /* iOSApp.swift */; }; + 1A00000000000000000002 /* ContentView.swift in Sources */ = {isa = PBXBuildFile; fileRef = 1A00000000000000000012 /* ContentView.swift */; }; + 1A00000000000000000003 /* Assets.xcassets in Resources */ = {isa = PBXBuildFile; fileRef = 1A00000000000000000013 /* Assets.xcassets */; }; +/* End PBXBuildFile section */ + +/* Begin PBXFileReference section */ + 1A00000000000000000010 /* iosApp.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = iosApp.app; sourceTree = BUILT_PRODUCTS_DIR; }; + 1A00000000000000000011 /* iOSApp.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = iOSApp.swift; sourceTree = ""; }; + 1A00000000000000000012 /* ContentView.swift */ = {isa = PBXFileReference; lastKnownFileType = sourcecode.swift; path = ContentView.swift; sourceTree = ""; }; + 1A00000000000000000013 /* Assets.xcassets */ = {isa = PBXFileReference; lastKnownFileType = folder.assetcatalog; path = Assets.xcassets; sourceTree = ""; }; + 1A00000000000000000014 /* Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.plist.xml; path = Info.plist; sourceTree = ""; }; +/* End PBXFileReference section */ + +/* Begin PBXFrameworksBuildPhase section */ + 1A00000000000000000020 /* Frameworks */ = { + isa = PBXFrameworksBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXFrameworksBuildPhase section */ + +/* Begin PBXGroup section */ + 1A00000000000000000030 /* Root */ = { + isa = PBXGroup; + children = ( + 1A00000000000000000031 /* iosApp */, + 1A00000000000000000032 /* Products */, + ); + sourceTree = ""; + }; + 1A00000000000000000031 /* iosApp */ = { + isa = PBXGroup; + children = ( + 1A00000000000000000011 /* iOSApp.swift */, + 1A00000000000000000012 /* ContentView.swift */, + 1A00000000000000000013 /* Assets.xcassets */, + 1A00000000000000000014 /* Info.plist */, + ); + path = iosApp; + sourceTree = ""; + }; + 1A00000000000000000032 /* Products */ = { + isa = PBXGroup; + children = ( + 1A00000000000000000010 /* iosApp.app */, + ); + name = Products; + sourceTree = ""; + }; +/* End PBXGroup section */ + +/* Begin PBXNativeTarget section */ + 1A00000000000000000040 /* iosApp */ = { + isa = PBXNativeTarget; + buildConfigurationList = 1A00000000000000000071 /* Build configuration list for PBXNativeTarget "iosApp" */; + buildPhases = ( + 1A00000000000000000023 /* Compile Kotlin Framework */, + 1A00000000000000000022 /* Sources */, + 1A00000000000000000020 /* Frameworks */, + 1A00000000000000000021 /* Resources */, + ); + buildRules = ( + ); + dependencies = ( + ); + name = iosApp; + productName = iosApp; + productReference = 1A00000000000000000010 /* iosApp.app */; + productType = "com.apple.product-type.application"; + }; +/* End PBXNativeTarget section */ + +/* Begin PBXProject section */ + 1A00000000000000000050 /* Project object */ = { + isa = PBXProject; + attributes = { + BuildIndependentTargetsInParallel = YES; + LastSwiftUpdateCheck = 1500; + LastUpgradeCheck = 1500; + TargetAttributes = { + 1A00000000000000000040 = { + CreatedOnToolsVersion = 15.0; + }; + }; + }; + buildConfigurationList = 1A00000000000000000070 /* Build configuration list for PBXProject "iosApp" */; + compatibilityVersion = "Xcode 14.0"; + developmentRegion = en; + hasScannedForEncodings = 0; + knownRegions = ( + en, + Base, + ); + mainGroup = 1A00000000000000000030 /* Root */; + productRefGroup = 1A00000000000000000032 /* Products */; + projectDirPath = ""; + projectRoot = ""; + targets = ( + 1A00000000000000000040 /* iosApp */, + ); + }; +/* End PBXProject section */ + +/* Begin PBXResourcesBuildPhase section */ + 1A00000000000000000021 /* Resources */ = { + isa = PBXResourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 1A00000000000000000003 /* Assets.xcassets in Resources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXResourcesBuildPhase section */ + +/* Begin PBXShellScriptBuildPhase section */ + 1A00000000000000000023 /* Compile Kotlin Framework */ = { + isa = PBXShellScriptBuildPhase; + buildActionMask = 2147483647; + files = ( + ); + inputFileListPaths = ( + ); + inputPaths = ( + ); + name = "Compile Kotlin Framework"; + outputFileListPaths = ( + ); + outputPaths = ( + ); + runOnlyForDeploymentPostprocessing = 0; + shellPath = /bin/sh; + shellScript = "cd \"$SRCROOT/..\"\n./gradlew :composeApp:embedAndSignAppleFrameworkForXcode\n"; + }; +/* End PBXShellScriptBuildPhase section */ + +/* Begin PBXSourcesBuildPhase section */ + 1A00000000000000000022 /* Sources */ = { + isa = PBXSourcesBuildPhase; + buildActionMask = 2147483647; + files = ( + 1A00000000000000000001 /* iOSApp.swift in Sources */, + 1A00000000000000000002 /* ContentView.swift in Sources */, + ); + runOnlyForDeploymentPostprocessing = 0; + }; +/* End PBXSourcesBuildPhase section */ + +/* Begin XCBuildConfiguration section */ + 1A00000000000000000060 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = dwarf; + ENABLE_STRICT_OBJC_MSGSEND = YES; + ENABLE_TESTABILITY = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_DYNAMIC_NO_PIC = NO; + GCC_NO_COMMON_BLOCKS = YES; + GCC_OPTIMIZATION_LEVEL = 0; + GCC_PREPROCESSOR_DEFINITIONS = ( + "DEBUG=1", + "$(inherited)", + ); + IPHONEOS_DEPLOYMENT_TARGET = 16.0; + MTL_ENABLE_DEBUG_INFO = INCLUDE_SOURCE; + MTL_FAST_MATH = YES; + ONLY_ACTIVE_ARCH = YES; + SDKROOT = iphoneos; + SWIFT_ACTIVE_COMPILATION_CONDITIONS = DEBUG; + SWIFT_OPTIMIZATION_LEVEL = "-Onone"; + }; + name = Debug; + }; + 1A00000000000000000061 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ALWAYS_SEARCH_USER_PATHS = NO; + CLANG_ANALYZER_NONNULL = YES; + CLANG_ANALYZER_NUMBER_OBJECT_CONVERSION = YES_AGGRESSIVE; + CLANG_ENABLE_MODULES = YES; + CLANG_ENABLE_OBJC_ARC = YES; + CLANG_WARN_DOCUMENTATION_COMMENTS = YES; + CLANG_WARN_UNGUARDED_AVAILABILITY = YES_AGGRESSIVE; + COPY_PHASE_STRIP = NO; + DEBUG_INFORMATION_FORMAT = "dwarf-with-dsym"; + ENABLE_NS_ASSERTIONS = NO; + ENABLE_STRICT_OBJC_MSGSEND = YES; + GCC_C_LANGUAGE_STANDARD = gnu11; + GCC_NO_COMMON_BLOCKS = YES; + IPHONEOS_DEPLOYMENT_TARGET = 16.0; + MTL_ENABLE_DEBUG_INFO = NO; + MTL_FAST_MATH = YES; + SDKROOT = iphoneos; + SWIFT_COMPILATION_MODE = wholemodule; + SWIFT_OPTIMIZATION_LEVEL = "-O"; + VALIDATE_PRODUCT = YES; + }; + name = Release; + }; + 1A00000000000000000062 /* Debug */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor; + CURRENT_PROJECT_VERSION = 1; + DEVELOPMENT_TEAM = ""; + ENABLE_PREVIEWS = YES; + GENERATE_INFOPLIST_FILE = YES; + INFOPLIST_FILE = iosApp/Info.plist; + INFOPLIST_KEY_UILaunchScreen_Generation = YES; + LD_RUNPATH_SEARCH_PATHS = ( + "$(inherited)", + "@executable_path/Frameworks", + ); + MARKETING_VERSION = 1.0; + PRODUCT_BUNDLE_IDENTIFIER = sk.ainet.samples.kernelrace; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 5.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Debug; + }; + 1A00000000000000000063 /* Release */ = { + isa = XCBuildConfiguration; + buildSettings = { + ASSETCATALOG_COMPILER_APPICON_NAME = AppIcon; + ASSETCATALOG_COMPILER_GLOBAL_ACCENT_COLOR_NAME = AccentColor; + CURRENT_PROJECT_VERSION = 1; + DEVELOPMENT_TEAM = ""; + ENABLE_PREVIEWS = YES; + GENERATE_INFOPLIST_FILE = YES; + INFOPLIST_FILE = iosApp/Info.plist; + INFOPLIST_KEY_UILaunchScreen_Generation = YES; + LD_RUNPATH_SEARCH_PATHS = ( + "$(inherited)", + "@executable_path/Frameworks", + ); + MARKETING_VERSION = 1.0; + PRODUCT_BUNDLE_IDENTIFIER = sk.ainet.samples.kernelrace; + PRODUCT_NAME = "$(TARGET_NAME)"; + SWIFT_VERSION = 5.0; + TARGETED_DEVICE_FAMILY = "1,2"; + }; + name = Release; + }; +/* End XCBuildConfiguration section */ + +/* Begin XCConfigurationList section */ + 1A00000000000000000070 /* Build configuration list for PBXProject "iosApp" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 1A00000000000000000060 /* Debug */, + 1A00000000000000000061 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; + 1A00000000000000000071 /* Build configuration list for PBXNativeTarget "iosApp" */ = { + isa = XCConfigurationList; + buildConfigurations = ( + 1A00000000000000000062 /* Debug */, + 1A00000000000000000063 /* Release */, + ); + defaultConfigurationIsVisible = 0; + defaultConfigurationName = Release; + }; +/* End XCConfigurationList section */ + }; + rootObject = 1A00000000000000000050 /* Project object */; +} diff --git a/KernelRace/iosApp/iosApp.xcodeproj/project.xcworkspace/contents.xcworkspacedata b/KernelRace/iosApp/iosApp.xcodeproj/project.xcworkspace/contents.xcworkspacedata new file mode 100644 index 0000000..919434a --- /dev/null +++ b/KernelRace/iosApp/iosApp.xcodeproj/project.xcworkspace/contents.xcworkspacedata @@ -0,0 +1,7 @@ + + + + + diff --git a/KernelRace/iosApp/iosApp.xcodeproj/xcshareddata/xcschemes/iosApp.xcscheme b/KernelRace/iosApp/iosApp.xcodeproj/xcshareddata/xcschemes/iosApp.xcscheme new file mode 100644 index 0000000..756f9df --- /dev/null +++ b/KernelRace/iosApp/iosApp.xcodeproj/xcshareddata/xcschemes/iosApp.xcscheme @@ -0,0 +1,50 @@ + + + + + + + + + + + + + + + + + + + diff --git a/KernelRace/iosApp/iosApp/Assets.xcassets/AccentColor.colorset/Contents.json b/KernelRace/iosApp/iosApp/Assets.xcassets/AccentColor.colorset/Contents.json new file mode 100644 index 0000000..eb87897 --- /dev/null +++ b/KernelRace/iosApp/iosApp/Assets.xcassets/AccentColor.colorset/Contents.json @@ -0,0 +1,11 @@ +{ + "colors" : [ + { + "idiom" : "universal" + } + ], + "info" : { + "author" : "xcode", + "version" : 1 + } +} diff --git a/KernelRace/iosApp/iosApp/Assets.xcassets/AppIcon.appiconset/Contents.json b/KernelRace/iosApp/iosApp/Assets.xcassets/AppIcon.appiconset/Contents.json new file mode 100644 index 0000000..13613e3 --- /dev/null +++ b/KernelRace/iosApp/iosApp/Assets.xcassets/AppIcon.appiconset/Contents.json @@ -0,0 +1,13 @@ +{ + "images" : [ + { + "idiom" : "universal", + "platform" : "ios", + "size" : "1024x1024" + } + ], + "info" : { + "author" : "xcode", + "version" : 1 + } +} diff --git a/KernelRace/iosApp/iosApp/Assets.xcassets/Contents.json b/KernelRace/iosApp/iosApp/Assets.xcassets/Contents.json new file mode 100644 index 0000000..73c0059 --- /dev/null +++ b/KernelRace/iosApp/iosApp/Assets.xcassets/Contents.json @@ -0,0 +1,6 @@ +{ + "info" : { + "author" : "xcode", + "version" : 1 + } +} diff --git a/KernelRace/iosApp/iosApp/ContentView.swift b/KernelRace/iosApp/iosApp/ContentView.swift new file mode 100644 index 0000000..aa66d29 --- /dev/null +++ b/KernelRace/iosApp/iosApp/ContentView.swift @@ -0,0 +1,17 @@ +import SwiftUI +import ComposeApp + +struct ComposeView: UIViewControllerRepresentable { + func makeUIViewController(context: Context) -> UIViewController { + MainViewControllerKt.MainViewController() + } + + func updateUIViewController(_ uiViewController: UIViewController, context: Context) {} +} + +struct ContentView: View { + var body: some View { + ComposeView() + .ignoresSafeArea(.all) + } +} diff --git a/KernelRace/iosApp/iosApp/Info.plist b/KernelRace/iosApp/iosApp/Info.plist new file mode 100644 index 0000000..f602a24 --- /dev/null +++ b/KernelRace/iosApp/iosApp/Info.plist @@ -0,0 +1,46 @@ + + + + + CFBundleDevelopmentRegion + en + CFBundleExecutable + $(EXECUTABLE_NAME) + CFBundleIdentifier + $(PRODUCT_BUNDLE_IDENTIFIER) + CFBundleInfoDictionaryVersion + 6.0 + CFBundleName + $(PRODUCT_NAME) + CFBundlePackageType + APPL + CFBundleShortVersionString + 1.0 + CFBundleVersion + 1 + LSRequiresIPhoneOS + + UIApplicationSceneManifest + + UIApplicationSupportsMultipleScenes + + + UILaunchScreen + + UIRequiredDeviceCapabilities + + arm64 + + UISupportedInterfaceOrientations + + UIInterfaceOrientationPortrait + + UISupportedInterfaceOrientations~ipad + + UIInterfaceOrientationPortrait + UIInterfaceOrientationPortraitUpsideDown + UIInterfaceOrientationLandscapeLeft + UIInterfaceOrientationLandscapeRight + + + diff --git a/KernelRace/iosApp/iosApp/iOSApp.swift b/KernelRace/iosApp/iosApp/iOSApp.swift new file mode 100644 index 0000000..768ceeb --- /dev/null +++ b/KernelRace/iosApp/iosApp/iOSApp.swift @@ -0,0 +1,11 @@ +import SwiftUI + +@main +struct iOSApp: App { + var body: some Scene { + WindowGroup { + ContentView() + .ignoresSafeArea(.keyboard) // Compose handles the keyboard itself + } + } +} diff --git a/KernelRace/shared/build.gradle.kts b/KernelRace/shared/build.gradle.kts index 0dc217d..edd2d65 100644 --- a/KernelRace/shared/build.gradle.kts +++ b/KernelRace/shared/build.gradle.kts @@ -12,6 +12,9 @@ kotlin { jvm() + iosArm64() + iosSimulatorArm64() + @OptIn(ExperimentalWasmDsl::class) wasmJs { browser() @@ -52,6 +55,16 @@ kotlin { } jvmMain.dependencies { implementation(libs.skainet.data.source) + // Transitive-only: lets Platform.jvm.kt resolve PanamaVectorKernelProvider's + // isAvailable() (declared on the KernelProvider interface from this module). + implementation(libs.skainet.backend.api) + } + iosMain.dependencies { + // skainet-data-source is JVM-only (skainet.targets=jvm in its gradle.properties) — + // KtorRemoteDataSourceFetcher isn't reachable from iosMain, so IosModelProvider + // talks to Ktor's Darwin engine directly instead. + implementation(libs.ktor.client.core) + implementation(libs.ktor.client.darwin) } commonTest.dependencies { implementation(libs.kotlin.test) diff --git a/KernelRace/shared/src/androidMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.android.kt b/KernelRace/shared/src/androidMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.android.kt index c8e28fc..ac900a5 100644 --- a/KernelRace/shared/src/androidMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.android.kt +++ b/KernelRace/shared/src/androidMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.android.kt @@ -16,6 +16,8 @@ actual fun kernelTierLabel(): String = actual val supportsKernelRace: Boolean = true +actual val currentSamplePlatform: SamplePlatform = SamplePlatform.ANDROID + /** ":scalar" vs main process, read once from /proc/self/cmdline — splits log tags so the * one-phone NEON-vs-scalar race can be watched as two separately filterable `adb logcat` streams. */ private val processTag: String by lazy { diff --git a/KernelRace/shared/src/commonMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.kt b/KernelRace/shared/src/commonMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.kt index 5b63003..23cc8ff 100644 --- a/KernelRace/shared/src/commonMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.kt +++ b/KernelRace/shared/src/commonMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.kt @@ -8,6 +8,18 @@ expect fun kernelTierLabel(): String /** Only Android can pin a JNI NEON provider vs a scalar one and race them side by side. */ expect val supportsKernelRace: Boolean +/** Every target this sample runs on — the fixed set the platform-chips row displays, + * independent of which one is currently running (see [currentSamplePlatform]). */ +enum class SamplePlatform(val label: String) { + ANDROID("Android"), + DESKTOP("Desktop"), + WEB("Web"), + IOS("iOS"), +} + +/** Which of [SamplePlatform] this process is actually running as — drives the highlighted chip. */ +expect val currentSamplePlatform: SamplePlatform + /** Where the platform actually writes a log line — `adb logcat` on Android, stdout elsewhere. */ internal expect fun platformLog(line: String) diff --git a/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/engine/LlamaRuntimeBuilder.ios.kt b/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/engine/LlamaRuntimeBuilder.ios.kt new file mode 100644 index 0000000..74135df --- /dev/null +++ b/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/engine/LlamaRuntimeBuilder.ios.kt @@ -0,0 +1,36 @@ +package sk.ainet.samples.kernelrace.engine + +import sk.ainet.apps.llm.OptimizedLLMMode +import sk.ainet.apps.llm.OptimizedLLMRuntime +import sk.ainet.apps.llm.tokenizer.TokenizerFactory +import sk.ainet.context.ExecutionContext +import sk.ainet.io.PosixPreadRandomAccessSource +import sk.ainet.io.model.QuantPolicy +import sk.ainet.lang.types.FP32 +import sk.ainet.models.llama.DecoderGgufWeightLoader +import sk.ainet.models.llama.LlamaNetworkLoader +import sk.ainet.samples.kernelrace.model.ModelData + +/** Same file-based random-access shape as the JVM/Android actuals, backed by POSIX `pread(2)` + * (shared by macOS/iOS/Linux native in skainet-io-core's native64Main) instead of a JDK/ART + * file API. `QuantPolicy.NATIVE_OPTIMIZED` keeps Q8_0 packed for the Apple native-cinterop + * kernels — see kllama's registerPlatformBackends()/installNativeKernels() wiring, which + * installs the same provider this app relies on. */ +actual suspend fun buildLlamaComponents(ctx: ExecutionContext, model: ModelData): LlamaComponents { + val path = (model as ModelData.FilePath).path + fun openSource() = checkNotNull(PosixPreadRandomAccessSource.open(path)) { "Cannot open GGUF at $path" } + val weights = DecoderGgufWeightLoader( + randomAccessProvider = { openSource() }, + quantPolicy = QuantPolicy.NATIVE_OPTIMIZED, + acceptedArchitectures = setOf("llama", "mistral"), // SmolLM2 is llama-family + ).loadToMapStreaming(ctx) + val runtime = OptimizedLLMRuntime( + model = LlamaNetworkLoader.fromWeights(weights), + ctx = ctx, + mode = OptimizedLLMMode.DIRECT, + dtype = FP32::class, + bos = weights.metadata.bosTokenId, + ) + val tokenizer = openSource().use { TokenizerFactory.fromGgufSource(it) } + return LlamaComponents(runtime, tokenizer) +} diff --git a/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/model/IosModelProvider.kt b/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/model/IosModelProvider.kt new file mode 100644 index 0000000..de3f461 --- /dev/null +++ b/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/model/IosModelProvider.kt @@ -0,0 +1,126 @@ +package sk.ainet.samples.kernelrace.model + +import io.ktor.client.HttpClient +import io.ktor.client.engine.darwin.Darwin +import io.ktor.client.plugins.HttpTimeout +import io.ktor.client.request.get +import io.ktor.client.request.header +import io.ktor.client.statement.bodyAsChannel +import io.ktor.http.HttpHeaders +import io.ktor.utils.io.asSource +import kotlin.time.TimeSource +import kotlinx.cinterop.ExperimentalForeignApi +import kotlinx.io.Buffer +import kotlinx.io.buffered +import kotlinx.io.files.Path +import kotlinx.io.files.SystemFileSystem +import platform.Foundation.NSBundle +import platform.Foundation.NSCachesDirectory +import platform.Foundation.NSFileManager +import platform.Foundation.NSSearchPathForDirectoriesInDomains +import platform.Foundation.NSUserDomainMask +import sk.ainet.samples.kernelrace.platform.logEvent + +/** + * No `skainet-data-source`/`AndroidModelProvider`-style bundled-asset copy step needed here in + * the same shape: iOS app bundles are read-only, so a bundled GGUF is used directly from + * `NSBundle.mainBundle` rather than copied into the cache dir first. Otherwise the same + * cache-hit-or-download shape as [DesktopModelProvider]. + */ +@OptIn(ExperimentalForeignApi::class) +class IosModelProvider( + private val cacheDir: String = defaultCacheDir(), +) : ModelProvider { + + override suspend fun resolve(onProgress: (String) -> Unit): ModelData { + SystemFileSystem.createDirectories(Path(cacheDir)) + val target = Path(cacheDir, HF_FILE) + + return when (val plan = ModelResolver.plan(target.toString(), targetExists(target), bundledAssetPath() != null)) { + is ResolutionPlan.UseCached -> { + logEvent("model_cached", "file" to HF_FILE, "fileMB" to sizeOf(target) / 1_000_000) + ModelData.FilePath(plan.path) + } + is ResolutionPlan.UseAsset -> { + val assetPath = checkNotNull(bundledAssetPath()) { "bundled asset disappeared between check and use" } + logEvent("model_from_bundle", "file" to HF_FILE) + ModelData.FilePath(assetPath) + } + ResolutionPlan.Download -> { + download(target, onProgress) + ModelData.FilePath(target.toString()) + } + } + } + + private fun bundledAssetPath(): String? = + NSBundle.mainBundle.pathForResource(HF_FILE.substringBeforeLast('.'), HF_FILE.substringAfterLast('.')) + + private fun targetExists(path: Path): Boolean = SystemFileSystem.exists(path) + + private fun sizeOf(path: Path): Long = SystemFileSystem.metadataOrNull(path)?.size ?: 0L + + private suspend fun download(target: Path, onProgress: (String) -> Unit) { + val url = "https://huggingface.co/$HF_REPO/resolve/main/$HF_FILE" + val tmp = Path(cacheDir, ModelResolver.partPath(HF_FILE)) + val client = HttpClient(Darwin) { + expectSuccess = true + install(HttpTimeout) { + requestTimeoutMillis = 600_000 + connectTimeoutMillis = 60_000 + socketTimeoutMillis = 600_000 + } + } + logEvent("download_start", "url" to url) + val startedAt = TimeSource.Monotonic.markNow() + var received = 0L + try { + val response = client.get(url) { header(HttpHeaders.Accept, "*/*") } + val totalMb = response.headers[HttpHeaders.ContentLength]?.toLongOrNull()?.let { it / 1_000_000 } + response.bodyAsChannel().asSource().buffered().use { source -> + SystemFileSystem.sink(tmp).buffered().use { sink -> + val chunk = Buffer() + var lastReported = -1L + while (true) { + val n = source.readAtMostTo(chunk, 1024 * 1024) + if (n == -1L) break + sink.write(chunk, n) + received += n + val mb = received / 1_000_000 + if (mb != lastReported) { + lastReported = mb + onProgress("Downloading model… $mb / ${totalMb ?: "?"} MB") + } + } + } + } + SystemFileSystem.atomicMove(tmp, target) + val durMs = startedAt.elapsedNow().inWholeMilliseconds + logEvent( + "download_done", + "fileMB" to received / 1_000_000, + "durMs" to durMs, + "MBps" to if (durMs > 0) (received / (durMs / 1000.0) / 1e6).toLong() else null, + ) + } catch (e: Exception) { + if (SystemFileSystem.exists(tmp)) SystemFileSystem.delete(tmp) + logEvent( + "download_failed", + "receivedMB" to received / 1_000_000, + "durMs" to startedAt.elapsedNow().inWholeMilliseconds, + "error" to (e.message ?: "unknown"), + ) + throw e + } finally { + client.close() + } + } + + companion object { + fun defaultCacheDir(): String { + val caches = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, true) + .firstOrNull() as? String + return "${caches ?: NSFileManager.defaultManager.currentDirectoryPath}/kernelrace/models" + } + } +} diff --git a/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.ios.kt b/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.ios.kt new file mode 100644 index 0000000..42b4290 --- /dev/null +++ b/KernelRace/shared/src/iosMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.ios.kt @@ -0,0 +1,18 @@ +package sk.ainet.samples.kernelrace.platform + +import platform.Foundation.NSLog + +/** No hardware-capability probe here (unlike Android's JniKernelProvider.isAvailable()) — + * the native-cinterop provider is a single archive with runtime FEAT_DotProd dispatch + * baked in, so there's no separate tier to report. */ +actual fun kernelTierLabel(): String = "Apple NEON (cinterop)" + +/** The two-process split-screen race is an Android-only mechanism (a second `:scalar` + * process re-pinning the kernel registry) — no equivalent process-spawn API on iOS. */ +actual val supportsKernelRace: Boolean = false + +actual val currentSamplePlatform: SamplePlatform = SamplePlatform.IOS + +internal actual fun platformLog(line: String) { + NSLog("SKAINET_PERF_IOS: %s", line) +} diff --git a/KernelRace/shared/src/jvmMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.jvm.kt b/KernelRace/shared/src/jvmMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.jvm.kt index efdde9c..80190cf 100644 --- a/KernelRace/shared/src/jvmMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.jvm.kt +++ b/KernelRace/shared/src/jvmMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.jvm.kt @@ -1,9 +1,25 @@ package sk.ainet.samples.kernelrace.platform -actual fun kernelTierLabel(): String = "JVM (scalar)" +import sk.ainet.exec.kernel.PanamaVectorKernelProvider + +/** + * Was hardcoded to "JVM (scalar)" — wrong whenever the Vector API incubator module is loaded + * (composeApp's :run task always adds --add-modules jdk.incubator.vector, see + * skainetSimdJvmArgs in composeApp/build.gradle.kts), which is the normal case. Probing + * PanamaVectorKernelProvider.isAvailable() directly — a static capability check, same + * pattern as Android's JniKernelProvider.isAvailable() — avoids depending on KernelRegistry + * being populated yet: ChatViewModel evaluates this at construction time, before + * DirectCpuExecutionContext.create() has run (which is what triggers registry population via + * KernelServiceLoader.installAll()), so a registry query here would always see an empty + * registry and report "scalar" regardless of what's actually about to run. + */ +actual fun kernelTierLabel(): String = + if (PanamaVectorKernelProvider.isAvailable()) "Panama Vector (SIMD)" else "JVM (scalar)" actual val supportsKernelRace: Boolean = false +actual val currentSamplePlatform: SamplePlatform = SamplePlatform.DESKTOP + internal actual fun platformLog(line: String) { println("[SKAINET_PERF_JVM] $line") } diff --git a/KernelRace/shared/src/wasmJsMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.wasmJs.kt b/KernelRace/shared/src/wasmJsMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.wasmJs.kt index 023dade..aee3bd7 100644 --- a/KernelRace/shared/src/wasmJsMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.wasmJs.kt +++ b/KernelRace/shared/src/wasmJsMain/kotlin/sk/ainet/samples/kernelrace/platform/Platform.wasmJs.kt @@ -4,6 +4,8 @@ actual fun kernelTierLabel(): String = "Wasm (scalar)" actual val supportsKernelRace: Boolean = false +actual val currentSamplePlatform: SamplePlatform = SamplePlatform.WEB + internal actual fun platformLog(line: String) { println("[SKAINET_PERF_WASM] $line") } diff --git a/KernelRace/webapp.json b/KernelRace/webapp.json index dd3fc4d..a3c56a7 100644 --- a/KernelRace/webapp.json +++ b/KernelRace/webapp.json @@ -2,7 +2,7 @@ "id": "kernelrace", "name": "Kernel Race", "description": "On-device LLM chat with a NEON-vs-scalar kernel race on Android; this web build runs SKaiNET's scalar fallback path.", - "platforms": ["android", "desktop", "wasm"], + "platforms": ["android", "desktop", "wasm", "ios"], "distDirs": [ "composeApp/build/dist/wasmJs/productionExecutable" ],