Lablr

   

Overview

Lablr is a web app that lets you drop in a product label photo and quickly get a plain-English readout of the ingredients, warnings, and safety notes. it's for anyone curious about what’s on the label.

Features

• Vision OCR agent that extracts ingredients, warnings, and claims with schema-constrained outputs.
• Automatic label language detection so OCR returns a BCP-47 language tag and downstream explanations localize accordingly.
• Explanation agent that merges local heuristics, glossary lookups, and optional OpenFoodFacts/OpenBeautyFacts fetches.
• Drag-and-drop web UI featuring progress tracking, risk badges, and structured results.
• File upload pipeline with automatic Interfaze hosting fallback to in-memory data URLs.
• TypeScript-first codebase with strict linting and shared utilities under @/tools and @ui/* aliases.

Demo Video

• https://youtu.be/pnJIVwg0mc8?si=yW3uEknqEa-Qx7vG
• 

Architecture

• agent/ocr.ts: Talks to Interfaze's vision completion endpoint, normalizes the payload, and reports the detected label language.
• agent/explain.ts: Generates ingredient summaries using local data (mcp/file-server) and optional external fetches, returning localized copy that matches the detected language.
• src/app: Next.js App Router entry point, API route, and UI wiring.
• ui/components: Reusable client components for upload, progress steps, and results tables.
• mcp: Minimal content provider assets (glossary, risk rules) served to the explanation agent.

Tech Stack

• Next.js App Router (React 19) with TypeScript and Tailwind CSS v4.
• Mastra 0.13.x agent runtime powering the multi-tool orchestration.
• Interfaze VOCR for high-quality vision language modeling.
• LibSQL bindings prepared for persistence experiments (optional).

Getting Started

1. Install dependencies

   pnpm install

2. Configure environment Create .env.local (UI) and .env (agent) as needed:

   INTERFAZE_API_KEY="sk-..."
   INTERFAZE_API_BASE="https://api.interfaze.ai/v1" # optional override
   INTERFAZE_OCR_MODEL="interfaze-beta"             # optional override
   WEB_FETCH_ENABLED="true"                         # enable external lookups
   OFF_FETCH_LIMIT="3"                              # cap external fetches per run

3. Run the stack

   pnpm dev:agent    # starts Mastra agent dev server (default :4111)
   pnpm dev:ui       # launches Next.js dev server (default :3000)

   Or run both with pnpm start once built.

Useful Scripts

• pnpm build: Compile agent + Next.js for production.
• pnpm start: Run both agent and UI in production mode.
• pnpm lint: Lint the TypeScript/React sources.

Testing The Flow

• Upload a product label image through the UI, or call POST /api/analyze with { "image_url": "..." }.
• Watch the console for warnings when environment variables are missing—agents gracefully downgrade to safe defaults.

Troubleshooting

• Empty OCR results: Ensure INTERFAZE_API_KEY is present; otherwise, the OCR agent returns a stub.
• Slow explanations: Disable WEB_FETCH_ENABLED to skip external data sources.
• Large images rejected: The API route limits uploads to 5 MB and common raster formats.

License

Distributed under the MIT License. See LICENSE for details.