SentinelStream

SentinelStream is a real-time Video Management System (VMS) with AI-powered person detection. It uses a modern event-driven architecture, streaming live video to browsers over low-latency WebRTC, publishing detection events via Redis Pub/Sub (Message Queue), and broadcasting updates to browser clients via WebSockets.

---

🏗️ Architecture Overview

The system consists of five dockerized services orchestrated by Docker Compose:

                   ┌──────────────────────────────────────────────┐
                   │                Browser (React)               │
                   │   ┌───────────────┐      ┌───────────────┐   │
                   │   │  Live Video   │      │  Alert Feed   │   │
                   │   └───────▲───────┘      └───────▲───────┘   │
                   └───────────┼──────────────────────┼───────────┘
                     WebRTC    │                      │ WebSocket
                 (Media Stream)│                      │ (Metadata)
                               │                      │
                       ┌───────┴──────┐        ┌──────┴───────┐
                       │    Worker    │        │   Backend    │
                       │   (Python)   │        │ (Bun + Hono) │
                       └───────┬──────┘        └──────▲───┬───┘
                               │                      │   │
                               │ Redis MQ             │   │
                               ▼                      │   │
                       ┌──────────────┐               │   │
                       │    Redis     ├───────────────┘   │
                       │  (Pub/Sub)   │                   │ PostgreSQL
                       └──────────────┘                   │
                                                          ▼
                                                   ┌──────────────┐
                                                   │  PostgreSQL  │
                                                   │   Database   │
                                                   └──────────────┘

1. Frontend (React + Vite + TS): A responsive dashboard featuring grid views for live cameras, real-time alert logs, custom status overlays, and complete CRUD control over RTSP cameras.
2. Backend (Bun + Hono + TS): Handles JWT-based authentication, camera configurations, alert history persistence, WebSocket client coordination, and subscribes to the Redis Pub/Sub MQ.
3. Worker (FastAPI + OpenCV + YOLOv8 + aiortc): Ingests RTSP streams, executes YOLOv8 object detection, mirror-flips frames, overlays OpenCV green bounding boxes, and streams H.264 video tracks over WebRTC directly to client browsers.
4. Redis (Message Queue): Decouples detection events from the HTTP pipeline, ensuring high-throughput ingestion and buffering.
5. MediaMTX (RTSP Simulator): Loops a test video file inside Docker to simulate IP security cameras.

---

💾 Unified Event Format

This JSON event payload format is preserved identically across the Python worker, Redis Pub/Sub, Postgres database, and the frontend WebSocket updates:

{
  "event_id": "0aeb9a21-8afd-4f91-8c21-65c3d37a5bc7",
  "camera_id": "b49a71ff-1219-46f1-883e-c38dc621fa03",
  "event_type": "person_detected",
  "timestamp": "2026-06-27T13:00:00.123Z",
  "confidence": 0.95,
  "bounding_box": {
    "x": 120,
    "y": 80,
    "width": 60,
    "height": 180
  },
  "frame_number": 4821,
  "thumbnail_url": null
}

• event_id: Unique UUID generated by the worker to enforce backend-side database deduplication.
• bounding_box: Normalised relative coordinates {x, y, width, height} of the detected person.
• thumbnail_url: Future extension for saving frame crops to cloud storage (e.g. AWS S3).

---

⚡ Tech Stack & Design Decisions

1. WebRTC Live Streaming

• aiortc (Python): Standard H.264 video track packetization ensures sub-second latency compared to HLS/DASH.
• Non-Trickle ICE: Both sides wait for candidate gathering to complete before the SDP handshake, simplifying signaling across isolated network bridges.

2. AI Person Detection & Worker Language

• YOLOv8n (Nano): The smallest variant of YOLOv8 is selected for CPU inference, executing in ~100-150ms.
• Frame Skipping: Inference is run every 10th frame (DETECT_EVERY_N_FRAMES=10) to prevent CPU exhaustion while keeping the input buffer clear.
• Python vs. Go: Python was chosen for the worker to leverage the native, optimized ultralytics package and aiortc binding, avoiding complex CGo configurations.
• OpenCV Drawing: Annotations are drawn directly on frames in memory on the worker. This guarantees absolute synchronization and avoids double-border overlays in React.

3. Decoupling & Queueing (Redis Pub/Sub)

• When YOLOv8 finds a person, it publishes the event to Redis channel sentinel:alerts.
• The backend subscribes to this channel. If Redis goes offline, the worker transparently falls back to direct HTTP POST alerts.

4. Deduplication & Rate Limiting

• Worker Cooldown: The worker maintains an in-memory rate-limiter, ensuring it only broadcasts one alert per camera every COOLDOWN_SECONDS (default: 15s).
• DB-Level Deduplication: The alerts table has a UNIQUE constraint on the event_id column. The backend executes ON CONFLICT (event_id) DO NOTHING during insertion. Duplicate posts resolve to 200 OK with { skipped: true } instead of raising exceptions.

---

🚀 How to Run

⚙️ Environment Configuration

Before launching the services, you must configure the environment variables:

1. Copy the .env.example file to create a .env file at the repository root:

   cp .env.example .env

2. Open .env and verify/configure the database credentials, ports, and internal worker secrets if needed. The default values are tuned to work out of the box with Docker Compose.

---

🐳 Option A: Run Everything via Docker Compose (Recommended)

This launches all five dockerized services (Postgres, Redis, MediaMTX, Backend, Worker, Frontend) automatically:

1. Make sure Docker Desktop is running.
2. Build and launch the containers:

   docker compose up -d --build

   (Subsequent runs can omit --build and start instantly with docker compose up -d).
3. Open your browser and navigate to:

   http://localhost:5173
   

4. Sign up a test account, log in, and view the simulated live feed!

---

💻 Option B: Local Development Setup (Outside Docker)

For active development, running frontend, backend, and worker directly on the host enables fast hot-reloads and debugging.

1. Start Postgres & Redis (via Docker Compose)

If you don't want to install Postgres and Redis servers locally, spin up just these infrastructure services:

docker compose up -d postgres redis mediamtx

2. Initialize the Database Schema

If running the Postgres service locally or outside of its initial container run, initialize the tables by executing the schema SQL script:

# If using the Docker Postgres service:
docker exec -i sentinel_postgres psql -U your_db_user -d your_db_name < backend/src/db/schema.sql

# If using a local Postgres installation:
psql -U your_db_user -d your_db_name -f backend/src/db/schema.sql

3. Run the Backend API

1. Navigate to the backend folder:

   cd backend

2. Install dependencies:

   bun install

3. Run the backend in development (watch) mode:

   bun run dev

   The backend API will start on http://localhost:3000.

4. Run the Frontend Dashboard

1. Navigate to the frontend folder:

   cd frontend

2. Install dependencies:

   npm install

3. Run the Vite development server:

   npm run dev

   The frontend UI will start on http://localhost:5173.

5. Run the Python Worker

1. Navigate to the worker folder:

   cd worker

2. Create and activate a Python virtual environment:

   python3 -m venv .venv
   source .venv/bin/activate

3. Install dependencies:

   pip install -r requirements.txt
   pip install "torch>=2.6.0" "torchvision>=0.21.0"

4. Run the FastAPI worker server:

   uvicorn main:app --host 0.0.0.0 --port 8001

---

📷 Run Local Webcam Stream (macOS / Host)

If you want to stream your own webcam as a live RTSP stream instead of the loop video:

1. Stop the worker container (if running via Docker):

   docker stop sentinel_worker

2. Run the worker locally (following the steps in Local Development Setup above).
3. Publish your webcam feed to MediaMTX using ffmpeg:

   ffmpeg -f avfoundation -framerate 30 -video_size 640x480 -i "0" \
     -pix_fmt yuv420p -vf "format=yuv420p" \
     -c:v libx264 -preset ultrafast -tune zerolatency \
     -f rtsp rtsp://localhost:8554/webcam

4. In the browser UI (Camera Manager), add a camera with the RTSP URL rtsp://localhost:8554/webcam.

---

🧪 Unit Tests

The Hono backend API endpoints are tested using Bun's built-in fast test runner, with mocked database queries and JWT assertions. Run the tests:

cd backend
bun run test

---

☸️ Kubernetes Deployment

Production-grade deployment configuration files are structured under infra/k8s/.

Deployment Steps

To deploy the SentinelStream stack onto a running Kubernetes cluster (e.g. Minikube, Docker Desktop K8s, or GKE):

1. Create the dedicated sentinelstream namespace:

   kubectl apply -f infra/k8s/namespace.yaml

2. Deploy the database and message broker:

   kubectl apply -f infra/k8s/postgres.yaml
   kubectl apply -f infra/k8s/redis.yaml

3. Deploy the MediaMTX simulator and wait for it to be ready:

   kubectl apply -f infra/k8s/mediamtx.yaml

4. Deploy the backend API and detection worker:

   kubectl apply -f infra/k8s/backend.yaml
   kubectl apply -f infra/k8s/worker.yaml

5. Verify that all components are running:

   kubectl get pods -n sentinelstream

---

🔮 Future Improvements

1. TURN Server Setup: Add a CoTURN container for WebRTC streaming outside local area networks.
2. GPU Acceleration: Deploy YOLOv8 onto CUDA/TensorRT runtime environments to process 30+ concurrent cameras at full frame rate.
3. Cloud Storage: Save event thumbnails to Amazon S3 buckets.
4. Horizontal Pod Autoscaling (HPA): Configure K8s HPA to scale worker pods dynamically based on CPU/GPU utilization spikes.