SentryBox

SentryBox turns an RTSP camera into a semantic Home Assistant event sensor. A watch combines a camera, an analysis region, a target definition, a schedule, and a local Ollama vision model. SentryBox captures frames with ffmpeg, asks Ollama for a schema-constrained verdict, applies deterministic debounce rules, and exposes the result as ordinary Home Assistant entities.

Local-only inference

SentryBox has no cloud inference client. Camera frames are sent only to the configured Ollama base URL, normally http://localhost:11434 or another host on the local network. No OpenAI, hosted-model, telemetry, or remote image-upload path is included.

The Ollama connection and model are configurable per watch. New entries default to:

• Base URL: SENTRYBOX_OLLAMA_BASE_URL or http://localhost:11434
• Model: SENTRYBOX_OLLAMA_MODEL or gemma3:4b

The selected Ollama model must support image input and JSON-schema output.

What a watch provides

• Package, trash-pickup, and custom target profiles
• RTSP/RTSPS capture with pipe and file fallbacks
• Shared raw-frame capture when multiple watches use one camera
• One-frame analysis for presence watches and a three-frame chronological burst for trash pickup
• Polygon analysis regions with an interactive preview
• Per-day schedules with multiple time windows
• detected, clear, uncertain, candidate, and cooldown runtime states
• Positive and negative debounce without treating uncertainty as a negative result
• Local event history and correct/incorrect feedback
• Latest analyzed-frame camera entity
• Administrator-only watch dashboard and write APIs

Architecture

The integration runs entirely inside Home Assistant:

schedule + monitoring switch
          ↓
shared frame broker → ffmpeg capture → polygon mask
          ↓
local Ollama /api/chat → strict result parser
          ↓
deterministic state machine → HA entities + local history

Important modules:

• analysis.py: strict structured-output parsing and legacy response compatibility
• providers/ollama.py: bounded retry and local Ollama request contract
• frame_broker.py: recent raw-frame sharing between watches
• state_machine.py: uncertainty-aware event transitions
• history.py: bounded Home Assistant Store event ledger
• coordinator.py: capture and analysis orchestration
• watch_api.py: administrator-only panel WebSocket API
• frontend/src/: typed Lit dashboard source
• www/sentrybox-panel.js: production panel bundle shipped through HACS

Configuration remains in Home Assistant config entries. Event metadata is stored in .storage/sentrybox.history. If snapshot retention is enabled, a single overwritten JPEG is stored at .sentrybox/<entry_id>.jpg; disabling retention removes that file while the current in-memory preview remains available.

Installation

1. Install and run Ollama on the Home Assistant host or a trusted local-network machine.
2. Pull a vision-capable model, for example ollama pull gemma3:4b.
3. Ensure ffmpeg is available on the Home Assistant host.
4. Copy custom_components/sentrybox into the Home Assistant configuration directory, or install the repository through HACS.
5. Restart Home Assistant.
6. Go to Settings → Devices & services → Add integration → SentryBox.
7. Enter the RTSP URL, local Ollama URL, model, and initial detection profile.
8. Open SentryBox in the sidebar to configure the region, schedule, policy, and review history.

Existing version 0.x entries migrate in place. Their stream URL, Ollama URL/model, profile, crop, schedule, prompt, entity unique IDs, service contract, and retained snapshot are preserved.

Detection output

Ollama receives a system instruction, the watch criteria, and one or more chronological JPEG frames. Its response is constrained to:

{
  "verdict": "detected",
  "confidence": 0.91,
  "summary": "A cardboard parcel is resting beside the front door.",
  "criteria": [
    {
      "id": "parcel_visible",
      "met": true,
      "evidence": "A sealed cardboard box is visible beside the door."
    }
  ],
  "quality": {
    "usable": true,
    "issues": []
  }
}

Malformed booleans, unusable images, invalid JSON, and unsupported verdicts become uncertain; they cannot turn a binary sensor on or clear an existing detection.

Home Assistant surface

Every watch exposes:

• A profile-aware detected binary sensor
• Confidence sensor
• Summary sensor
• Monitoring switch
• Last Analysis Frame camera

Shared attributes include the verdict, analysis state, criteria evidence, quality issues, model, frame count, latency, schedule, threshold, debounce streaks, and last error.

The existing sentrybox.reanalyze_now action remains supported. Its optional entry_id targets one watch; omitting it schedules all loaded watches.

Compatibility REST endpoints remain available to administrators:

• GET /api/sentrybox/schedule
• POST /api/sentrybox/schedule/{entry_id}
• GET /api/sentrybox/crop
• POST /api/sentrybox/crop/{entry_id}
• POST /api/sentrybox/crop/{entry_id}/preview

The new panel uses administrator-only Home Assistant WebSocket commands under sentrybox/watch/*.

Development

Python checks:

PYTHONPYCACHEPREFIX=/tmp/sentrybox-pycache python3 -m compileall -q custom_components/sentrybox
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s tests -p 'test_*.py'

Frontend checks and production bundle:

npm install
npm run check
npm test
npm run build

Tests cover crop and schedule normalization, strict model-output parsing, malformed legacy output, uncertainty preservation, debounce transitions, prompt/schema contracts, frontend geometry, and panel cache busting. A real Home Assistant test instance with a representative camera and local Ollama model is still required to evaluate visual accuracy and hardware-specific latency.

License

Copyright 2026 Esau Engineering. Licensed under the Apache License, Version 2.0. See LICENSE.