A powerful, cross-platform benchmarking suite designed to evaluate the performance of various Deep Learning inference engines across Mobile (Android & iOS) and Desktop (Windows, macOS, Linux) using Kotlin Multiplatform.
The ML landscape is fragmented. Developers often struggle to decide which engine to use: Should I stick with TFLite for ease of use, or integrate NCNN for raw performance?
KMPMLBench provides a unified "Lab" to test these scenarios with identical models and logic, minimizing the "overhead" of platform-specific implementations.
| Engine | Android | iOS | Desktop (JVM/Native) | Acceleration | Status |
|---|---|---|---|---|---|
| ONNX Runtime | 🏗️ | 🏗️ | 🏗️ | CoreML, DirectML, XNNPACK | In Progress |
| TensorFlow Lite | 📅 | 📅 | 📅 | NNAPI, CoreML, GPU | Planned |
| NCNN | 📅 | 📅 | 📅 | Vulkan, Metal | Planned |
| MNN | 📅 | 📅 | 📅 | OpenCL, Vulkan, Metal | Planned |
| ExecuTorch | 📅 | 📅 | 📅 | XNNPACK, CoreML | Planned |
While the project started with Super-Resolution, it is designed to be modular:
- Super-Resolution: ESPCN, FSRCNN, Real-ESRGAN.
- Image Classification: MobileNetV3, EfficientNet (Planned).
- Object Detection: YOLOv8-Nano (Planned).
- On-device LLM: Gemma 2B / Phi-2 (Experimental via ExecuTorch).
We measure more than just speed:
- Inference Latency: Average time per execution (ms).
- Initialization Time: Cold start vs. warm start (ms).
- Memory Peak: Maximum RAM/VRAM usage (MB).
- Energy Consumption: (Mobile only) Battery impact during prolonged tasks.