Add CompreFace face detection service with DoubleTake integration

README.md

@@ -10,7 +10,7 @@ This project was 99% developed by AI assistants (Gemini and GitHub Copilot). The
 
 * **Host OS:** Optimized for **openSUSE Leap Micro** (or other transactional OS) for enhanced stability and rollback capability.
 * **Container Runtime:** Uses **Podman** for managing containers, networks, and persistent volumes.
-* **Core Components:** Integrates **MQTT Broker** (Mosquitto), **Time Series Database** (InfluxDB), **Visualization** (Grafana), **Automation** (Node-RED), **NVR** (Frigate with Double-Take facial recognition), and **Zigbee Gateway** (Zigbee2MQTT).
+* **Core Components:** Integrates **MQTT Broker** (Mosquitto), **Time Series Database** (InfluxDB), **Visualization** (Grafana), **Automation** (Node-RED), **NVR** (Frigate with Double-Take facial recognition using CompreFace), and **Zigbee Gateway** (Zigbee2MQTT).
 * **Reverse Proxy:** Nginx-based reverse proxy with hostname-based routing for all services, including openSUSE Cockpit web console. Nginx configuration is dynamically generated based on running services to prevent startup failures.
 * **Security:** Uses `create_secrets.sh` to generate unique, random, 64-character passwords/tokens for sensitive environment variables.
 * **External Storage:** Includes logic to mount an **SMB/CIFS** share for Frigate recordings on the host machine.
@@ -92,7 +92,7 @@ The script will ask you to choose between:
 
 1. **IoT/SCADA Stack only** - Includes: Mosquitto (MQTT Broker), InfluxDB (Time Series Database), Grafana (Visualization), Node-RED (Automation), and Zigbee2MQTT (Zigbee Gateway)
 
-2. **NVR only** - Includes: Frigate (Network Video Recorder for camera management and object detection) and Double-Take (facial recognition)
+2. **NVR only** - Includes: Frigate (Network Video Recorder for camera management and object detection), Double-Take (facial recognition), and CompreFace (face detection service)
 
 3. **Both IoT/SCADA Stack + NVR** - Includes all services from both options above
 
@@ -112,15 +112,41 @@ PODMAN_SOCKET_PATH=/run/user/$(id -u)/podman/podman.sock
 ```
 
 * Other site-specific variables like `TZ` (timezone), `SMB_SERVER`, `SMB_SHARE`, `SMB_USER` (if using NVR), etc.
-* Nginx reverse proxy hostnames: `BASE_DOMAIN`, `GRAFANA_HOSTNAME`, `FRIGATE_HOSTNAME`, `NODERED_HOSTNAME`, `ZIGBEE2MQTT_HOSTNAME`, `COCKPIT_HOSTNAME`, `DOUBLETAKE_HOSTNAME`
+* Nginx reverse proxy hostnames: `BASE_DOMAIN`, `GRAFANA_HOSTNAME`, `FRIGATE_HOSTNAME`, `NODERED_HOSTNAME`, `ZIGBEE2MQTT_HOSTNAME`, `COCKPIT_HOSTNAME`, `DOUBLETAKE_HOSTNAME`, `COMPREFACE_HOSTNAME`
 
 ### 3. Configure Frigate (NVR Only)
 
 If you selected the NVR option, you need to configure Frigate:
 
 * Edit the `frigate_config.yml` file to define your cameras and settings.
 
-### 4. Run the Stack
+### 4. Configure CompreFace and Double-Take (NVR Only)
+
+If you selected the NVR option, the stack includes **CompreFace** for face detection, which integrates with **Double-Take** for facial recognition:
+
+**CompreFace Setup:**
+
+1. After running `./startup.sh`, CompreFace will be accessible at `http://compreface.<BASE_DOMAIN>` or `http://<host_ip>:8000`
+2. On first access, you'll need to create an admin account through the CompreFace web interface
+3. After logging in, create an API key:
+   - Go to the CompreFace dashboard
+   - Create a new application (e.g., "DoubleTake")
+   - Create a new recognition service within that application
+   - Copy the API key for the recognition service
+4. Update the `COMPREFACE_API_KEY` in your `secrets.env` file with the API key you just created (replace the auto-generated placeholder)
+5. Restart the Double-Take container: `./startup.sh start doubletake`
+
+**Double-Take Configuration:**
+
+Double-Take is pre-configured to work with CompreFace. Once CompreFace is set up with a valid API key, Double-Take will automatically use it for face detection. You can:
+
+- Access Double-Take at `http://doubletake.<BASE_DOMAIN>` or `http://<host_ip>:3001`
+- Add face images through the Double-Take interface to train recognition
+- Configure detection settings and notifications as needed
+
+**Note:** CompreFace requires approximately 2-4GB of RAM. Ensure your system meets the 8GB minimum requirement for NVR mode.
+
+### 5. Run the Stack
 
 After completing the manual configuration in `secrets.env`, run the setup again:
 
@@ -169,7 +195,7 @@ To troubleshoot or manually start a specific service:
 # Example: ./startup.sh start zigbee2mqtt
 ```
 
-Available service names: `mosquitto`, `influxdb`, `zigbee2mqtt`, `frigate`, `grafana`, `nodered`, `nginx`, `doubletake`.
+Available service names: `mosquitto`, `influxdb`, `zigbee2mqtt`, `frigate`, `grafana`, `nodered`, `nginx`, `doubletake`, `compreface`, `compreface_postgres`.
 
 **Changing Stack Configuration**
 
@@ -226,7 +252,7 @@ SERVICE_NAMES=(mosquitto influxdb zigbee2mqtt frigate grafana nodered nginx doub
 If you want the service to be properly cleaned up when running `./startup.sh breakdown`, add it to the `CONTAINER_NAMES` array in the `breakdown_containers_only()` function (around line 445):
 
 ```bash
-CONTAINER_NAMES=("mosquitto" "zigbee2mqtt" "frigate" "influxdb" "grafana" "nodered" "nginx" "doubletake" "codesysgateway")
+CONTAINER_NAMES=("mosquitto" "zigbee2mqtt" "frigate" "influxdb" "grafana" "nodered" "nginx" "doubletake" "compreface" "compreface_postgres" "codesysgateway")
 ```
 
 ### Step 3: (Optional) Configure Nginx Proxy
@@ -272,6 +298,7 @@ podman logs codesysgateway
 | **Grafana** | Data Visualization (SCADA UI) | http://grafana.&lt;BASE_DOMAIN&gt; or :3000 | IoT/SCADA modes only |
 | **Frigate** | NVR and Object Detection | http://frigate.&lt;BASE_DOMAIN&gt; or :5000 | NVR modes only |
 | **Double-Take** | Facial Recognition for Frigate | http://doubletake.&lt;BASE_DOMAIN&gt; or :3001 | NVR modes only |
+| **CompreFace** | Face Detection Service | http://compreface.&lt;BASE_DOMAIN&gt; or :8000 | NVR modes only, used by Double-Take |
 | **Node-RED** | Flow-Based Automation | http://nodered.&lt;BASE_DOMAIN&gt; or :1880 | IoT/SCADA modes only |
 | **Zigbee2MQTT** | Zigbee Device Control | http://zigbee.&lt;BASE_DOMAIN&gt; or :8080 | IoT/SCADA modes only |
 | **Cockpit** | openSUSE Web Console | http://cockpit.&lt;BASE_DOMAIN&gt; | Requires Cockpit enabled on host |

create_secrets.sh

@@ -42,6 +42,8 @@ sed -e "
     /^GRAFANA_ADMIN_PASSWORD=/c\GRAFANA_ADMIN_PASSWORD=$(generate_random_string)
     /^GRAFANA_SECRET_KEY=/c\GRAFANA_SECRET_KEY=$(generate_random_string)
     /^SMB_PASS=/c\SMB_PASS=$(generate_random_string)
+    /^COMPREFACE_API_KEY=/c\COMPREFACE_API_KEY=$(generate_random_string)
+    /^POSTGRES_PASSWORD=/c\POSTGRES_PASSWORD=$(generate_random_string)
 " "$ENV_EXAMPLE_FILE" > "$TEMP_FILE"
 
 # The 'c\' command in sed completely replaces the line with the specified string.

secrets.env-example

@@ -32,7 +32,8 @@ GRAFANA_ADMIN_PASSWORD=your_secure_grafana_password
 GRAFANA_SECRET_KEY=a_long_random_string_for_security_and_sessions
 
 # NODE-RED CONFIGURATION
-TZ=Europe/Berlin # Adjust timezone as needed
+# Adjust timezone as needed
+TZ=Europe/Berlin
 NODERED_PORT=1880
 
 # NGINX REVERSE PROXY CONFIGURATION
@@ -57,3 +58,13 @@ SMB_SERVER=my.smbserver.home
 SMB_SHARE=frigate_share
 SMB_USER=username
 SMB_PASS=secret_token_for_smb
+
+# COMPREFACE CONFIGURATION
+COMPREFACE_PORT=8000
+COMPREFACE_API_KEY=your_secure_compreface_api_key
+COMPREFACE_HOSTNAME=compreface
+
+# COMPREFACE DATABASE (PostgreSQL)
+POSTGRES_USER=compreface_admin
+POSTGRES_PASSWORD=your_secure_postgres_password
+POSTGRES_DB=compreface

startup.sh

@@ -27,6 +27,8 @@ VOLUME_LIST=(
     "influxdb_data"
     "nginx_cache"
     "doubletake_data"
+    "compreface_db_data"
+    "compreface_data"
 )
 # Array to track the startup status of each service
 declare -A SERVICE_STATUS
@@ -75,6 +77,12 @@ NODERED_HOSTNAME=$(read_var NODERED_HOSTNAME)
 ZIGBEE2MQTT_HOSTNAME=$(read_var ZIGBEE2MQTT_HOSTNAME)
 COCKPIT_HOSTNAME=$(read_var COCKPIT_HOSTNAME)
 DOUBLETAKE_HOSTNAME=$(read_var DOUBLETAKE_HOSTNAME)
+COMPREFACE_HOSTNAME=$(read_var COMPREFACE_HOSTNAME)
+COMPREFACE_PORT=$(read_var COMPREFACE_PORT)
+COMPREFACE_API_KEY=$(read_var COMPREFACE_API_KEY)
+POSTGRES_USER=$(read_var POSTGRES_USER)
+POSTGRES_PASSWORD=$(read_var POSTGRES_PASSWORD)
+POSTGRES_DB=$(read_var POSTGRES_DB)
 
 
 # ----------------------------------------------------------------------
@@ -410,6 +418,31 @@ NGINX_EOF
         echo "  [INFO] Double-Take is not running - skipping from nginx config"
     fi
 
+    # Check and add CompreFace if running
+    if echo "$running_services" | grep -q "^compreface$"; then
+        echo "  [ok] CompreFace is running - adding to nginx config"
+        cat >> "${nginx_conf_file}" << NGINX_EOF
+
+    # CompreFace (Face Detection Service)
+    server {
+        listen 80;
+        server_name ${COMPREFACE_HOSTNAME}.${BASE_DOMAIN};
+
+        location / {
+            proxy_pass http://compreface:8080;
+            proxy_set_header Host \$host;
+            proxy_set_header X-Real-IP \$remote_addr;
+            proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
+            proxy_set_header X-Forwarded-Proto \$scheme;
+            client_max_body_size 10M;
+        }
+    }
+NGINX_EOF
+        services_html+="<li><a href=\"http://${COMPREFACE_HOSTNAME}.${BASE_DOMAIN}\">CompreFace</a></li>"
+    else
+        echo "  [INFO] CompreFace is not running - skipping from nginx config"
+    fi
+
     # Check and add InfluxDB if running
     if echo "$running_services" | grep -q "^influxdb$"; then
         echo "  [ok] InfluxDB is running (available on port 8086)"
@@ -637,6 +670,12 @@ check_first_run() {
             ZIGBEE2MQTT_HOSTNAME=$(read_var ZIGBEE2MQTT_HOSTNAME)
             COCKPIT_HOSTNAME=$(read_var COCKPIT_HOSTNAME)
             DOUBLETAKE_HOSTNAME=$(read_var DOUBLETAKE_HOSTNAME)
+            COMPREFACE_HOSTNAME=$(read_var COMPREFACE_HOSTNAME)
+            COMPREFACE_PORT=$(read_var COMPREFACE_PORT)
+            COMPREFACE_API_KEY=$(read_var COMPREFACE_API_KEY)
+            POSTGRES_USER=$(read_var POSTGRES_USER)
+            POSTGRES_PASSWORD=$(read_var POSTGRES_PASSWORD)
+            POSTGRES_DB=$(read_var POSTGRES_DB)
         fi
     fi
 }
@@ -695,7 +734,7 @@ mount_smb_share() {
 # --- Breakdown function: Stop and Remove all containers (KEEP volumes) ---
 breakdown_containers_only() {
     echo "Stopping and removing containers..."
-    CONTAINER_NAMES=("mosquitto" "zigbee2mqtt" "frigate" "influxdb" "grafana" "nodered" "nginx" "doubletake")
+    CONTAINER_NAMES=("mosquitto" "zigbee2mqtt" "frigate" "influxdb" "grafana" "nodered" "nginx" "doubletake" "compreface" "compreface_postgres")
 
     for name in "${CONTAINER_NAMES[@]}"; do
         if podman ps -a --format '{{.Names}}' | grep -q "^${name}$"; then
@@ -747,8 +786,10 @@ SERVICE_CMDS[frigate]="podman run -d --name frigate --restart unless-stopped --n
 SERVICE_CMDS[grafana]="podman run -d --name grafana --restart unless-stopped --network ${NETWORK_NAME} -p 3000:3000 -v grafana_data:/var/lib/grafana -e GF_SECURITY_ADMIN_USER=${GRAFANA_ADMIN_USER} -e GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_ADMIN_PASSWORD} -e GF_SECURITY_SECRET_KEY=${GRAFANA_SECRET_KEY} docker.io/grafana/grafana:latest"
 SERVICE_CMDS[nodered]="podman run -d --name nodered --restart unless-stopped --network ${NETWORK_NAME} -p ${NODERED_PORT}:1880 -e TZ=${TZ} -e DOCKER_HOST=unix:///var/run/docker.sock -v nodered_data:/data -v ${PODMAN_SOCKET_PATH}:/var/run/docker.sock:ro --security-opt label=disable --user root docker.io/nodered/node-red:latest"
 SERVICE_CMDS[nginx]="podman run -d --name nginx --restart unless-stopped --network ${NETWORK_NAME} --add-host=host.containers.internal:host-gateway -p 80:80 --security-opt label=disable -v ${PWD}/nginx/nginx.conf:/etc/nginx/nginx.conf:ro -v nginx_cache:/var/cache/nginx docker.io/library/nginx:alpine"
-SERVICE_CMDS[doubletake]="podman run -d --name doubletake --restart unless-stopped --network ${NETWORK_NAME} -p 3001:3000 -v doubletake_data:/.storage -e TZ=${TZ} docker.io/jakowenko/double-take:latest"
-SERVICE_NAMES=(mosquitto influxdb zigbee2mqtt frigate grafana nodered nginx doubletake)
+SERVICE_CMDS[doubletake]="podman run -d --name doubletake --restart unless-stopped --network ${NETWORK_NAME} -p 3001:3000 -v doubletake_data:/.storage -e TZ=${TZ} -e DETECTORS__COMPREFACE__URL=http://compreface:8080 -e DETECTORS__COMPREFACE__KEY=${COMPREFACE_API_KEY} -e MQTT__HOST=mosquitto -e MQTT__USERNAME=${MQTT_USER} -e MQTT__PASSWORD=${MQTT_PASSWORD} -e FRIGATE__URL=http://frigate:5000 docker.io/jakowenko/double-take:latest"
+SERVICE_CMDS[compreface_postgres]="podman run -d --name compreface_postgres --restart unless-stopped --network ${NETWORK_NAME} -v compreface_db_data:/var/lib/postgresql/data -e POSTGRES_USER=${POSTGRES_USER} -e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} -e POSTGRES_DB=${POSTGRES_DB} docker.io/library/postgres:15"
+SERVICE_CMDS[compreface]="podman run -d --name compreface --restart unless-stopped --network ${NETWORK_NAME} -p ${COMPREFACE_PORT}:8080 -v compreface_data:/home/app/frs -e POSTGRES_USER=${POSTGRES_USER} -e POSTGRES_PASSWORD=${POSTGRES_PASSWORD} -e POSTGRES_URL=jdbc:postgresql://compreface_postgres:5432/${POSTGRES_DB} -e ENABLE_EMAIL_SERVER=false -e ADMIN_JAVA_OPTS=-Xmx4g -e MAX_FILE_SIZE=10MB -e MAX_REQUEST_SIZE=10MB docker.io/exadel/compreface:latest"
+SERVICE_NAMES=(mosquitto influxdb zigbee2mqtt frigate grafana nodered nginx doubletake compreface_postgres compreface)
 
 # --- Manual Start Function ---
 start_manual_service() {
@@ -770,7 +811,19 @@ start_manual_service() {
             exit 1
         fi
 
-        if [ "$SERVICE_NAME" != "frigate" ] && [ "$SERVICE_NAME" != "doubletake" ] && [ "$stack_type" == "nvr_only" ]; then
+        if [ "$SERVICE_NAME" == "compreface" ] && [ "$stack_type" == "iot_only" ]; then
+            echo "ERROR: CompreFace is not enabled in your configuration (IoT/SCADA only mode)."
+            echo "To enable CompreFace, delete ${CONFIG_FILE} and run ./startup.sh to reconfigure."
+            exit 1
+        fi
+
+        if [ "$SERVICE_NAME" == "compreface_postgres" ] && [ "$stack_type" == "iot_only" ]; then
+            echo "ERROR: CompreFace PostgreSQL is not enabled in your configuration (IoT/SCADA only mode)."
+            echo "To enable CompreFace, delete ${CONFIG_FILE} and run ./startup.sh to reconfigure."
+            exit 1
+        fi
+
+        if [ "$SERVICE_NAME" != "frigate" ] && [ "$SERVICE_NAME" != "doubletake" ] && [ "$SERVICE_NAME" != "compreface" ] && [ "$SERVICE_NAME" != "compreface_postgres" ] && [ "$stack_type" == "nvr_only" ]; then
             echo "ERROR: ${SERVICE_NAME} is not enabled in your configuration (NVR only mode)."
             echo "To enable IoT/SCADA services, delete ${CONFIG_FILE} and run ./startup.sh to reconfigure."
             exit 1
@@ -853,8 +906,19 @@ setup_system() {
             SERVICE_STATUS["${SERVICE}"]="SKIPPED (Not configured)"
             continue
         fi
-        # Skip IoT services if stack type is nvr_only (but keep frigate and doubletake)
-        if [ "$SERVICE" != "frigate" ] && [ "$SERVICE" != "doubletake" ] && [ "$stack_type" == "nvr_only" ]; then
+        # Skip CompreFace and PostgreSQL if stack type is iot_only
+        if [ "$SERVICE" == "compreface" ] && [ "$stack_type" == "iot_only" ]; then
+            echo "Skipping CompreFace (NVR not enabled in configuration)"
+            SERVICE_STATUS["${SERVICE}"]="SKIPPED (Not configured)"
+            continue
+        fi
+        if [ "$SERVICE" == "compreface_postgres" ] && [ "$stack_type" == "iot_only" ]; then
+            echo "Skipping CompreFace PostgreSQL (NVR not enabled in configuration)"
+            SERVICE_STATUS["${SERVICE}"]="SKIPPED (Not configured)"
+            continue
+        fi
+        # Skip IoT services if stack type is nvr_only (but keep frigate, doubletake, compreface, and compreface_postgres)
+        if [ "$SERVICE" != "frigate" ] && [ "$SERVICE" != "doubletake" ] && [ "$SERVICE" != "compreface" ] && [ "$SERVICE" != "compreface_postgres" ] && [ "$stack_type" == "nvr_only" ]; then
             echo "Skipping $SERVICE (IoT/SCADA not enabled in configuration)"
             SERVICE_STATUS["${SERVICE}"]="SKIPPED (Not configured)"
             continue