Skip to content

AndroidCraft — on-device generative AI studio (chat, image, voice, video) + CI/CD APK - #1

Merged
Ishuin merged 6 commits into
mainfrom
claude/android-generative-ai-app-n9po39
Aug 5, 2026
Merged

Ishuin merged 6 commits into
mainfrom
claude/android-generative-ai-app-n9po39

Conversation

@Ishuin

@Ishuin Ishuin commented Aug 5, 2026

Copy link
Copy Markdown
Owner

What this is

AndroidCraft — a fully on-device generative AI app for Android (Kotlin, Jetpack Compose, MVVM). Chat, generate images, synthesize speech, and make short video clips — all running locally on the phone, with models you download in-app. Every APK is built in CI, never on a device or laptop.

main is seeded at the initial scaffold commit; this PR contains the full build-out on top of it (media export, the image engine selector, real native + ONNX diffusion, and the curated model catalog).

Features

Tab Runs on-device
Chat Offline LLM inference via Google AI Edge / MediaPipe GenAI (LiteRT .task)
Image Method selector: Procedural (instant) · Diffusion · SD.cpp (native stable-diffusion.cpp) · Diffusion · ONNX (ONNX Runtime, shown when a bundle is present)
Voice Text-to-speech → WAV via the system neural TTS engine
Video Prompt-seeded motion clip → real H.264 MP4 (MediaCodec/MediaMuxer)
Models Curated one-tap downloads, sorted by device RAM fit, with a "Best for your device" badge

Every generated image, voice clip, and video can be saved (gallery / Music / Movies) and shared.

Curated models — zero manual setup

The Models screen offers only vetted, ungated, direct-download models — no account, no Hugging Face token, no links to find, no manual URL entry. They're matched to the device's reported RAM so the user doesn't have to guess what fits.

  • Chat: Gemma 3 1B (int4) · Qwen 2.5 1.5B (int8) — the same LiteRT models Google's AI Edge Gallery ships.
  • Image: Stable Diffusion 1.5 as single-file GGUF (Q8) for the native diffusion engine — one tap to download and run.

Real on-device diffusion

  • stable-diffusion.cpp is vendored as a submodule under app/src/main/cpp and compiled for arm64 via NDK/CMake; sdjni.cpp is a thin JNI bridge. The whole pipeline (tokenizer, scheduler, UNet, VAE) runs in C++.
  • ONNX Runtime engine implements the full SD 1.5 pipeline in Kotlin (CLIP byte-level BPE tokenizer + UNet with classifier-free guidance + DDIM scheduler + VAE) on onnxruntime-android.

CI/CD

  • .github/workflows/android-build.yml — on every build-affecting commit: inits native submodules, sets up JDK 17 + Android SDK + NDK 27 / CMake, runs unit tests, compiles the native library, assembles the debug APK, and uploads it as the androidcraft-debug-apk artifact.
  • .github/workflows/release.yml — on a v* tag, builds and publishes a GitHub Release with the APK (auto-signs if keystore secrets are set).

Notes / honest caveats

  • The app builds and links cleanly in CI. The diffusion engines are compile-verified in CI but not device-tested from the build environment — the first real generation on-device is the true test. Each engine is gated behind "is the model present," so a mismatch shows an error on that one tab rather than crashing the app.
  • Toolchain: AGP 8.7.3 / Kotlin 2.0.21 / compileSdk 35 / minSdk 26 / arm64.
  • Clone with --recurse-submodules for local builds.

🤖 Generated with Claude Code


Generated by Claude Code

claude added 6 commits August 4, 2026 19:30
- Token is now fully optional: sent only when provided, so ungated models
  download with no Hugging Face login.
- Add-model-by-URL: users can add any direct LiteRT .task chat model from the
  Models screen; custom models persist on-device and appear across the app.
- Downloadable/shareable outputs:
  - Image: save to gallery + share (MediaStore).
  - Voice: save WAV to Music + share.
  - Video: encode a real H.264 MP4 on-device (MediaCodec + MediaMuxer),
    save to Movies + share.
- Reactive model list in ModelRepository so custom models update the UI live.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQL2UkPX1shJkQqSBrV8A7
- New ImageGenerator abstraction with selectable on-device backends:
  Procedural (instant), stable-diffusion.cpp (native), ONNX Runtime.
- Image screen gains a method picker, per-method availability hints, a
  sampling-steps slider, and diffusion progress.
- Diffusion engines are structured and report readiness; procedural works
  now, with the native and ONNX engines wired in follow-up commits.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQL2UkPX1shJkQqSBrV8A7
- Add stable-diffusion.cpp as a submodule + JNI bridge (sdjni) building the
  core diffusion library for arm64 via CMake/NDK. The full pipeline
  (tokenizer, scheduler, UNet, VAE) runs in C++, fully offline.
- NativeDiffusionGenerator now calls the native txt2img and is available once
  a Stable Diffusion .gguf/.safetensors model is downloaded.
- Catalog: add downloadable Stable Diffusion 1.5 weights; Add-by-URL can now
  target image (SD) models too.
- CI installs the NDK + CMake and initialises the native submodules.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQL2UkPX1shJkQqSBrV8A7
- OnnxDiffusionGenerator: full SD 1.5 pipeline on ONNX Runtime (CLIP text
  encoder + UNet with classifier-free guidance + VAE decoder), fully offline.
- ClipTokenizer (byte-level BPE) and a deterministic DDIM scheduler in Kotlin.
- ModelRepository gains .zip archive download + extraction for multi-file
  bundles; catalog adds the SD 1.5 ONNX bundle.
- Adds the onnxruntime-android dependency (arm64).

Both diffusion engines (native SD.cpp and ONNX) are now selectable in the
Image tab, each activating once its model is downloaded.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQL2UkPX1shJkQqSBrV8A7
- Replace manual URL entry + Hugging Face token flow with a curated catalog of
  vetted, ungated, direct-download models (verified against Google's AI Edge
  Gallery allowlist and single-file GGUF repos).
- Chat: Gemma 3 1B (int4) + Qwen 2.5 1.5B (int8), no account required.
- Image: Stable Diffusion 1.5 Q8 GGUF (single-file) for the native engine —
  one tap to download and run.
- Models screen sorts each category by device-RAM fit and badges the best pick
  "Best for your device"; removes the token card and add-by-URL dialog.
- Image picker hides the ONNX engine unless its bundle is present; native
  engine picks the most recently downloaded model.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PQL2UkPX1shJkQqSBrV8A7
@Ishuin
Ishuin merged commit 4380e09 into main Aug 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants