NotesAI

An Android notes app that answers questions about your notes, with the whole pipeline running on the phone: embedding, vector search, generation and tool calling. No network required once the model is downloaded.

It is also a learning project, and that is the part that shaped it. Every milestone is a git tag, and every tag has a decision log entry explaining what problem forced the change and what the fix cost. The full write-up is on Medium: On-Device RAG on Android: A Step-by-Step Build Guide.

If you landed here without reading that: the app started as the naive version, where every note was pasted into the prompt and sent to Gemini, and ended up with a 2.6 GB model on the phone's CPU choosing which of four tools to call. Each tag between those two points is one step of that road.

---

What it does

• Add, list and delete notes
• Ask questions about them in natural language
• Answers come from an agent that picks its own tools: search_notes, add_note, count_notes, get_all_notes
• Runs fully on device when the model is available, falls back to cloud Gemini when it isn't
• Every answer is labelled with where it actually came from, device or cloud

Architecture

UI (Compose)
  └─ ViewModel (Hilt, StateFlow, UDF)
       └─ AgentSelector  ────────────────► implements Agent
            ├─ OnDeviceAgent    (LiteRT-LM + Gemma 4, tool calling)
            └─ NotesAgent       (Firebase AI Logic + Gemini, function calling)
                 └─ NotesRepository
                      ├─ ObjectBox        (notes + HNSW vector index)
                      ├─ NoteEmbedder     (MediaPipe Text Embedder)
                      ├─ OnDeviceTextGenerator (LiteRT-LM)
                      └─ ModelDownloader  (DownloadManager)

Agent has four implementations. Three of them answer questions: the cloud agent, the on-device agent, and OnDeviceRagAnswerer, a fixed non-agentic pipeline (topK = 3, no tools). The fourth is AgentSelector, which implements the same interface and picks between the first two, so the ViewModel only ever sees one Agent.

OnDeviceRagAnswerer is deliberately left in the codebase and not wired into the selector. It is the baseline arm of the evaluation.

Stack

Layer	Choice
Language / UI	Kotlin 2.0.21, Jetpack Compose, Material 3
DI	Hilt 2.59.2 (KSP)
Local storage + vector search	ObjectBox 5.4.2, @HnswIndex(dimensions = 100, distanceType = COSINE) (kapt)
Embedding	MediaPipe Text Embedder 0.10.29 + Universal Sentence Encoder, 100-dim, English only
On-device LLM	LiteRT-LM 0.13.1 + Gemma-4-E2B-it, Backend.CPU()
Cloud LLM	Firebase AI Logic (BOM 34.14.0), gemini-2.5-flash
Model download	Android DownloadManager, in-app
Build	AGP 9.0.1, Gradle 9.1.0, KSP 2.3.9, kapt (ObjectBox only)
SDK	minSdk 26, targetSdk 36, compileSdk 36

The project runs both annotation processors: KSP for Hilt, kapt for ObjectBox, which has no KSP support yet (objectbox-java#1075).

Requirements

JDK 21 to run the build. AGP 9 / Gradle 9 need a modern JDK regardless of the app's own Java target (11).

A physical Android device. Not optional. MediaPipe, ObjectBox and LiteRT-LM all ship native libraries, and the MediaPipe one crashes on the emulator with SIGILL because the virtual CPU does not implement the instructions it was compiled for.

Roughly 3 GB of free storage for the on-device model, and 8 GB of device RAM or more. Below 8 GB the app shows a warning before downloading but does not block you, matching the threshold Google's AI Edge Gallery declares for this model.

Your own Firebase project for the cloud path. app/google-services.json is not in this repository. Create a Firebase project, enable Firebase AI Logic with the Gemini Developer API backend, and drop the file in app/. The free tier is enough.

Developed and measured on a Pixel 10 Pro XL (Tensor G5, 16 GB). No claims are made about other devices.

Assets and models

What	Size	Where it comes from
universal_sentence_encoder.tflite	6.1 MB	committed, in app/src/main/assets/
gemma-4-E2B-it.litertlm	~2.6 GB	downloaded by the app on first use

The model is fetched from Hugging Face (litert-community/gemma-4-E2B-it-litert-lm) at a pinned commit hash, into the app's external files directory, using a temporary filename that is renamed only after the download completes. Until it is there, the selector uses the cloud path.

Build and run

git clone <repo>
cd NotesAIApp
# put your google-services.json in app/
./gradlew installDebug

Unit tests (JVM, no device needed):

./gradlew testDebugUnitTest

Running the evaluation

The evaluation compares the agentic and non-agentic on-device arms over a fixed 12-note fixture, measuring context recall and which tool the model chose. 14 primary questions plus two cross-language checks, every agentic measurement repeated three times. Methodology and results: docs/evaluation.md.

It needs the model already downloaded on the device. The measured run took 12 and a half minutes.

./gradlew installDebug installDebugAndroidTest

adb logcat -c && adb logcat -s NotesAiEval > eval-run.log &

adb shell am instrument -w \
  -e class com.caner.notesai.eval.OnDeviceEvalRun \
  com.caner.notesai.test/androidx.test.runner.AndroidJUnitRunner

adb pull /storage/emulated/0/Android/data/com.caner.notesai/files/eval-results.md .

Do not run this with ./gradlew connectedDebugAndroidTest. That task uninstalls both APKs when it finishes, and uninstalling wipes /Android/data/com.caner.notesai, which takes the results file and the 2.6 GB model with it.

This is a measurement run, not a test. It asserts nothing and cannot fail the build; it uses the instrumentation runner purely as a way to execute code on the device against the real model.

Milestones

Each tag is a working checkpoint, and the decision log at that tag contains the reasoning as it stood then, including plans that were later abandoned.

tag	what changed
journey-1-naive-rag	every note goes into the prompt, Room storage, Firebase AI Logic
journey-2-on-device-rag	MediaPipe embeddings, hand-written cosine, top-3 retrieval
journey-3-objectbox-vector-db	Room replaced by ObjectBox, HNSW index inside the database
journey-4-agentic-function-calling	Gemini function calling, four tools, model-driven retrieval
journey-4.1-agentic-hardening	max-turn guard, input guards, Agent interface, unit tests
journey-5-on-device-generation	LiteRT-LM + Gemma 4, in-app model download, cloud fallback
journey-6-on-device-agentic	on-device tool calling, on-device instruction, few-shot example

The tags stop at journey-6. The evaluation and the final write-up live on main.

Docs

File	What's in it
docs/decision-log.md	every significant decision and why, written at the time
docs/evaluation.md	evaluation methodology, fixture, question set, results
docs/eval-run-2026-08-07.log	raw log from the measured run

License

Apache License 2.0. See LICENSE.