Smart Wardrobe

A privacy-first wardrobe assistant, with the ML built from scratch.

Take photos of your clothes. Smart Wardrobe finds each garment, labels it (category, colour, pattern, sleeve length, fit…), turns it into a clean catalog image, suggests outfits, picks what to wear today based on the weather and your calendar, and lets you chat with a stylist. Later, virtual try-on.

Status: early. Phase 0 (API skeleton) is in place, Phase 1 (garment understanding) is in progress.

Approach

• Own models. Every step (detection, attributes, catalog image, compatibility, ranking, try-on) is a model I train, evaluate and serve. Hosted APIs and zero-shot models are only baselines to beat on a golden test set.
• Privacy by design. Photos stay on self-hosted models. Consent, encryption and deletion are built in from the start.
• No agent where none is needed. Core features are fixed pipelines behind plain endpoints. An LLM with tool calling is used only for the stylist chat.

Full reasoning in docs/ROADMAP.md.

Architecture

            ┌────────────── mobile app ──────────────┐
            │                                         │
            ▼                                         ▼
   services/api (FastAPI)  ───── stylist chat ───► LLM (tool calling)
            │                                         │
            │  contract.py (the only ML ↔ backend link)│ tools = same endpoints
            ▼                                         ▼
   ┌─────────────┬───────────────┬──────────────┬─────────────┬──────────┐
   │ /v1/analyze │ catalog image │ compatibility│ recommender │  try-on  │
   └─────────────┴───────────────┴──────────────┴─────────────┴──────────┘
                     one model per endpoint, trained in ml/

The backend and the ML only meet in services/api/app/ml/contract.py. Until a real model is ready, the API runs against a stub analyzer.

Roadmap

Phase	Feature	Key metrics	Status
0	Foundations: repo, API skeleton, tracking	—	in progress
1	Garment understanding	mask mAP, per-attribute macro-F1, recall@k	in progress
2	Shop-style catalog image	human preference, FID	planned
3	Outfits + occasion	compatibility AUC, FITB accuracy, occasion F1	planned
4	Daily recommendation	NDCG@k, hit rate, acceptance rate	planned
5	Stylist chat	task success on scripted eval	planned
6	Virtual try-on	human eval, FID/KID	planned

Phase 1 notes: docs/phase1-garment-understanding.md.

Layout

Path	What
ml/	Taxonomy and dataset configs, loaders, notebooks, training, evaluation
services/api/	Backend (FastAPI, SQLModel)
docs/	Roadmap, phase notes, decisions and dataset licenses

Quick start

Requires Python 3.12+ and uv.

git config core.hooksPath .githooks
cd services/api && uv run uvicorn app.main:app --reload   # http://localhost:8000/docs
uv run pytest

Set WARDROBE_ANALYZER=http to use the real model service instead of the stub. ML setup is in ml/README.md.

License

Code is Apache 2.0. Datasets and pretrained weights keep their own licenses, logged in docs/decisions.md.