**FoodFinder** AI-powered & barcode food identification for carb entry#2404
Open
taylorpatterson-T1D wants to merge 254 commits intoLoopKit:devfrom
Open
**FoodFinder** AI-powered & barcode food identification for carb entry#2404taylorpatterson-T1D wants to merge 254 commits intoLoopKit:devfrom
taylorpatterson-T1D wants to merge 254 commits intoLoopKit:devfrom
Conversation
…, analysis history - Fix triple barcode fire by consuming scan result immediately in Combine sink - Replace AsyncImage with pre-downloaded thumbnail to avoid SwiftUI rebuild issues - Use smallest OFF thumbnail (100px) with static food icon fallback for slow servers - Add secure Keychain storage for AI provider API keys - Add analysis history tracking with FoodFinder_AnalysisRecord - Consolidate AI provider settings and remove BYOTestConfig
- Remove barcode connectivity pre-check that added 3+ seconds latency per scan - Add NSCache to ImageDownloader for thumbnail deduplication (50 items, 10MB) - Remove artificial minimumSearchDuration delay from search and error paths - Merge duplicate Combine observers into single combineLatest for AI recomputation - Decode image_thumb_url from OpenFoodFacts API for smallest available thumbnail - Wrap 369 bare print() calls in #if DEBUG across 8 FoodFinder files
…eaders File consolidations (6 files removed, 2 new files created): 1. FoodFinder_ScanResult.swift + FoodFinder_VoiceResult.swift → FoodFinder_InputResults.swift 2. FoodFinder_FavoriteDetailView.swift + FoodFinder_FavoriteEditView.swift + FoodFinder_FavoritesView.swift → FoodFinder_FavoritesHelpers.swift 3. FoodFinder_AISettingsManager.swift → absorbed into FoodFinder_AIProviderConfig.swift 4. FoodFinder_FavoritesViewModel.swift → absorbed into FoodFinder_SearchViewModel.swift Other changes: - Fix long analysis titles overflowing the screen by programmatically truncating picker row names and constraining food type to 20 chars - Improve AI prompts for menu/recipe/text image analysis - Add text-only AI analysis path in AIServiceManager - Increase AI token budget for multi-item responses - Standardize all 26 FoodFinder file headers with consistent format
- Add originalAICarbs and aiConfidencePercent fields to FoodFinder_AnalysisRecord for tracking AI estimate accuracy - Add Notification.Name.foodFinderMealLogged for real-time meal event observation - Add MealDataProvider protocol with date-range query interface and AnalysisHistoryStore conformance - Add "Last 30 days" retention option to Analysis History settings
- Add originalAICarbs and aiConfidencePercent fields to FoodFinder_AnalysisRecord for tracking AI estimate accuracy - Add Notification.Name.foodFinderMealLogged for real-time meal event observation - Add MealDataProvider protocol with date-range query interface and AnalysisHistoryStore conformance - Add "Last 30 days" retention option to Analysis History settings
- Absorption time model: conservative adjustments anchored to Loop's 3-hour default. FPU adds +0/+0.5/+1.0 hr (was +1/+2.5/+4), fiber +0/+0.25/+0.5 (was +0/+1/+2), meal size +0/+0.25/+0.5 (was +0/+1/+2). Cap reduced from 8 to 5 hours. Updated AI prompt and 3 examples. - OCR routing fix: raised menu detection threshold from 1 to 5 significant lines and always include image on menu path to prevent food photo misclassification (fixes "Unidentifiable Food Item" on food photos). - Inline "Why X hrs?" pill on Absorption Time row replaces standalone DisclosureGroup row. Purple centered pill with fixed width, expands reasoning on tap. Uses AIAbsorptionTimePickerRow when AI-generated.
taylorpatterson-T1D
changed the title
Add LoopInsights feature: an AI-driven therapy settings advisor that analyzes glucose, insulin, and carb data to suggest adjustments to Carb Ratios, Insulin Sensitivity Factors, and Basal Rates. Core components: - Dashboard with therapy settings overview, pattern detection, and AI suggestions - Configurable AI provider (OpenAI, Anthropic, Gemini, Grok, self-hosted) - Data aggregation pipeline with test data fixtures from Tidepool - Suggestion lifecycle: pending → applied/dismissed with full history - AI personality settings (Supportive Coach, Clinical Expert, Dry Wit, Tough Love) - Developer mode with auto-apply and test data toggles - Secure API key storage via Keychain - Safety guardrails: max 20% change per adjustment, one setting at a time - Unit tests for models, data aggregation, and suggestion store 22 new files, 4 modified files across Views, View Models, Models, Services, Managers, Resources, and Tests.
…ng, and UI refinements - Wire real therapy settings writes via LoopInsightsSettingsWriter closure - Schedule splitting: insert new entries when AI suggests times not in user's schedule - Revert feature: restore pre-apply settings from suggestion history - Settings Score (0-100) with TIR, Safety, Stability, GMI breakdown - Clinical reasoning framework: AI now understands AID-specific patterns (corrections/day, basal/bolus ratio, time-of-day analysis, cross-setting interactions) - All three settings visible in every AI prompt for cross-setting reasoning - Pre-computed red flags injected into prompt (algorithm workload, basal % alerts) - Stale-data guard: excludes manually reverted changes from recent context - Suggestion merge: consolidates split AI responses into single cards - Pre-Fill Editor: editable proposed values before applying - Auto-applied notification banner - Debug log with Copy Full Log for troubleshooting AI behavior - Temperature forced to 0.0 for deterministic analysis
… advisor UI Add Ask LoopInsights chat with AI advisor powered by therapy context and glucose data. Background monitoring with configurable frequency and notification banners. New Trends & Insights view with Daily/Weekly/Monthly/Stats/Advisor tabs. Dark gradient styling for chat and trends views. Banner now includes Ask button to open chat directly.
…ports Add clinical goal tracking (TIR, A1C, below-range, custom) with progress bars, AI-powered 30-day pattern discovery with sick day and negative basal detection, timestamped reflection journal with mood tags, and HTML-to-PDF report generation with share sheet. Goals & Patterns accessible from the Dashboard navigation section.
… analysis Add HealthKit biometric data (heart rate, HRV, steps, sleep, active energy, weight) to the AI analysis and chat pipelines. Biometrics are read-only, independently authorized, and gracefully degrade when individual types are unavailable. New file: LoopInsights_HealthKitManager.swift Modified: Models, DataAggregator, AIAnalysis, ChatViewModel, Coordinator, FeatureFlags, SettingsView, DashboardView, pbxproj, Localizable.xcstrings
…nsights, Nightscout import - Ambulatory Glucose Profile (AGP) chart with percentile bands and median line - Clarity-style dashboard redesign: Glucose card, Time in Range 5-zone stacked bar, capsule period picker with exact Clarity colors (#C14F0C, #F0CA4C, #74A52E, #D36265, #7F0302) - Caffeine tracker with half-life decay modeling and glucose correlation - Meal insights with food response analysis and per-meal glucose impact - Nightscout data import support - Advanced analyzers for pattern detection - 5-zone TIR breakdown (Very High/High/In Range/Low/Very Low) replacing 3-zone model - Compact list section spacing for tighter dashboard layout - Chat view UI refinements
…card fixes P1: Parallel HealthKit queries via async let (6 concurrent fetches) P2: Single-pass TIR zone counting (5-zone) replacing multiple filter passes P3: Pre-fetch raw data in DataAggregator, cache for cross-component reuse P4: Binary search for glucose lookups in FoodResponseAnalyzer P5: Pre-sorted glucose samples with binary search in AdvancedAnalyzers P6: Pre-compute AGP data in ViewModel instead of SwiftUI view body P7: Static DateFormatter in LoopInsightsTimeBlock.formatTime P8: Pre-sort schedule items before dose loops, pre-sort in ViewModel P9: Pre-convert glucose to parallel arrays avoiding repeated doubleValue calls P10: Pass precomputed hourly averages to circadian profile builder Also: enhanced step/activity data in AI prompts with time-of-day breakdowns and activity-glucose correlation analysis (2h lag), and meal card layout cleanup. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…y fixes Glucose chart now operates in two modes: standard Ambulatory Glucose Profile (24-hour overlay with percentile bands) for 14-day lookback, and Glucose Profile (multi-day time series) for all other periods. Both modes include an info button explaining the visualization. HealthKit glucose data supplements Loop store for longer analysis periods. Chart data clears on period change to prevent stale labels. Additional fixes across 22 files: improved HealthKit data pipeline reliability, enhanced test data provider, refined food response analysis, and minor bug fixes in background monitor, coordinator, caffeine tracker, and goals/trends views. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…y fixes Glucose chart now operates in two modes: standard Ambulatory Glucose Profile (24-hour overlay with percentile bands) for 14-day lookback, and Glucose Profile (multi-day time series) for all other periods. Both modes include an info button explaining the visualization. HealthKit glucose data supplements Loop store for longer analysis periods. Chart data clears on period change to prevent stale labels. Additional fixes across 22 files: improved HealthKit data pipeline reliability, enhanced test data provider, refined food response analysis, and minor bug fixes in background monitor, coordinator, caffeine tracker, and goals/trends views.
Bump all body text, headers, and stat values to full white for readability on dark backgrounds. Replace .toolbarColorScheme (iOS 16+) with manual toolbar principal title for compatibility. Restore UINavigationBarAppearance approach in ChatView. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
taylorpatterson-T1D
changed the title
This was referenced Feb 14, 2026
taylorpatterson-T1D
force-pushed
the
feat/FoodFinder
branch
from
4e9b9ff to
ab2a8cf
Compare
The 20-char limit was truncating food names (e.g. "Baked pastry with f…") which made them unreadable in LoopInsights Meal Insights. The RowEmojiTextField maxLength only restricts keyboard input, so longer programmatic values are safe. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
taylorpatterson-T1D
force-pushed
the
feat/FoodFinder
branch
from
ab2a8cf to
63c53fc
Compare
Added steps for creating and using test data in developer mode for demos and feature functionality testing.
…ivity CoreMotion-based activity detection that automatically applies user-selected override presets when walking or running is detected. 7 new files, 2 modified. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…data Analyzes FoodFinder meal history for systematic user corrections by food type, location, time of day, and AI confidence level. Surfaces patterns as alerts on the dashboard and in a dedicated Behavior Insights view. Injects patterns into AI supplemental context and Post-Meal debrief prompts for richer recommendations. Also makes Settings Impact section collapsible via DisclosureGroup.
taylorpatterson-T1D
changed the title
taylorpatterson-T1D
changed the title
…ights Adds one-tap PDF report for endocrinologist appointments with branded LoopInsights teal header, toggleable sections, vertical Dexcom Clarity-style Time in Range bar, detected glucose/insulin patterns, behavior correction patterns, and custom email subject line. Report covers glucose, insulin, nutrition, settings changes, biometrics, engagement, caffeine/alcohol, and pump suspensions.
…very Configurable digest for caregivers with recipient email/phone storage, delivery method picker (Email or iMessage), frequency (daily/weekly), and personalized greeting. Send Now generates summary and opens pre-filled compose view — recipient just taps Send. Includes TIR, glucose stats, insulin delivery, and meal data.
Automatically activate presets when arriving at or leaving saved locations (gym, office, park) using iOS region monitoring. Battery-efficient — no continuous GPS tracking. Includes map picker, radius config, trigger type selection, and full integration with existing AutoPresets activity log.
EventKit integration that scans calendars for keyword-matched events and auto-activates presets with configurable lead time. Supports all calendar providers, per-calendar filtering, and deactivation on event end.
Replaces the existing minimal chart touch highlight with a rich detail popup showing glucose, IOB, COB, bolus, basal, preset, AutoPreset, and heart rate data at any point on the glucose chart. Supports scrubbing left/right with haptic feedback, auto-fades after 5 seconds, respects safe areas in both orientations, and works standalone with optional enhanced data when other features (AutoPresets, etc.) are enabled.
Only on feat/AllFeatures — individual feature branches keep stock Loop icon. Replaces all 18 tracked PNGs in DerivedAssetsBase with purple PowerPack icon.
This reverts commit 008877b.
This reverts commit 8f4a0de.
Track where you place insulin pump infusion sets and CGM sensors on a visual body map. Color-coded pins show site age (red = fresh, green = safe to reuse). Prompted logging on pump deactivation, manual logging anytime. Drag-and-drop pin adjustment with proximity-based coloring warns when placing near recently-used sites. 11 new files, 3 modified files, ~1,829 lines added.
- 12 recommended placement zones as grey ellipses on body map - Toggleable zones via collapsible DisclosureGroup in Settings - Pin expands to 110pt with halo on touch-drag - Proximity-based color (green=safe, red=danger) during drag - Body bounds clamping prevents pins outside silhouette - Save button pinned to bottom of selection sheet - Feature toggle takes effect immediately - Updated body map images and app icon - iOS 15 compatible (.strokeBorder instead of .stroke)
Loose PNGs in Resources/SiteAtlas/ weren't being included in the app
bundle despite correct pbxproj registration. Moved to imagesets in
DerivedAssetsBase.xcassets and simplified loading to Image("name").
CLGeocoder often returns shopping center names instead of the specific restaurant. Added MKLocalSearch refinement that finds the closest food venue within 100m and replaces the generic geocode name. Also updated the AI prompt to include the restaurant name in the food title.
"Get help with Therapy Settings" now opens LoopInsights instead of the blank DemoPlaceHolderView stub when LoopInsights is enabled.
onAppear on therapySettingsView fired too late — at the same instant TherapySettingsView rendered its body, so the registry was still nil. Moving it to SettingsView's top-level body ensures it fires when the user opens Settings, before they navigate to Therapy Settings.
…ghts and AutoPresets Internal data + thresholds stay canonical mg/dL; conversion happens at the display + AI-prompt boundary via a new LoopInsights_GlucoseUnitContext helper. AI prompts get a unit-context block so Claude responds in the user's unit; canonical JSON fields (target_range_*_mgdl, ISF current/proposed_value) stay mg/dL by explicit prompt instruction. LoopInsights_Coordinator.init now requires displayGlucosePreference; the dataStoresProvider tuple is extended from 5 to 6 elements. AutoPresets_Coordinator and PreMealAdvisorService get the preference set during boot in LoopAppManager. Display surfaces fixed: Dashboard, Trends, Meal Insights, Meal Debrief, Endo Report, Caregiver Digest (text + HTML), Goals report, Suggestion Detail, Settings tight-range stepper, AutoPresets override editor. AI prompts updated: ChatViewModel, AIAnalysis (with mg/dL JSON-field rule for ISF values), TrendsInsightsView, GoalsView, MealInsightsViewModel, AutoPresets_AIAdvisor (with mg/dL JSON-field rule for target ranges).
A pizza on a paper menu was being identified off the menu text instead of the visible plate. Three fixes: - Require text bounding boxes to cover >75% of the image before routing through the menu-text path (previously: 5+ lines of any size triggered) - Reframe menu-path prompt so visible food stays primary; OCR text is only used to identify the dish or restaurant - Hoist LOCATION CONTEXT to the top of every prompt with REQUIRED venue-in-title and 📍-prefix rules that apply across image_types
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
FoodFinder adds AI-powered food identification to Loop's carb entry screen, helping people with diabetes quickly and accurately log meals. It integrates directly into the existing Add Carb Entry flow with zero changes to Loop's core dosing logic. Part of the Loop AI PowerPack.
The Problem We're Solving:
Carb counting is the single hardest daily task for people managing diabetes with Loop. Every meal requires estimating carbohydrate content — and getting it wrong directly impacts Time in Range. Current workflow: the user mentally estimates carbs, types a number, and hopes for the best. There's no assistance, no database lookup, no learning from past meals.
What FoodFinder Does
FoodFinder adds AI-powered food identification directly into Loop's existing Add Carb Entry screen. It provides four ways to identify food and auto-populate carb values:
Search modes:
Key features:
FoodFinder_FeatureFlags.isEnabled) — completely dormant when disabledUser-Configurable Settings
All settings are in Loop Settings → FoodFinder:
gpt-4o,claude-sonnet-4-5-20250929,gemini-2.0-flash) (Be sure you use models supporting image processing)Safety Considerations
Architecture and impact on existing Loop code
FoodFinder was designed for minimal integration footprint and easy containment within Loop:
FoodFinder/subdirectories, all prefixedFoodFinder_Modified existing files:
CarbEntryView.swiftFoodFinder_EntryPoint(~5 lines) + analysis history pickerSettingsView.swiftCarbEntryViewModel.swiftFavoriteFoodDetailView.swiftFavoriteFoodsView.swiftAddEditFavoriteFoodView.swiftAddEditFavoriteFoodViewModel.swiftNew file locations (all under
Loop/):Models/FoodFinder/View Models/FoodFinder/Views/FoodFinder/Services/FoodFinder/Resources/FoodFinder/Documentation/FoodFinder/LoopTests/FoodFinder/Screenshots
Video Demo
YouTube Demo: https://youtu.be/i8xToAYBe4M
Test plan
Requesting review by @marionbarker based on availability.
Recent Updates (since initial PR)
Per-Item Portion Control
Individual USDA serving steppers for each food item in AI results. Replaces the single plate-level multiplier — nutrition circles update live as you adjust individual items.
AI Performance Optimizations
VNDetectTextRectanglesRequestfast gate before expensive OCR — skips full OCR when <5 text regions found (most food photos aren't menus)Location-Aware AI Analysis
GPS reverse geocode integration — AI prompt receives restaurant/venue name when available for more accurate food identification. Location cached per-session, cleared on dismiss.
Image Crop Step
Users can crop photos before AI analysis for more focused, accurate results.
Macro-Aware Absorption Time (ungated)
Extended absorption time calculation for high fat/protein meals — raised high-FPU adjustment and cap. No core dosing algorithm changes; uses Loop's existing extended absorption natively.
Pre-Bolus Timing Recommendations
AI now always includes timing recommendation based on GI category, meal composition, and current glucose context.
AI Carb Range Display
Shows confidence range (e.g. "42–58g") alongside the point estimate to help users calibrate expectations.
Analysis History Improvements