🛡️ DefendPulse: Real-Time SIEM & Threat Monitoring Console

DefendPulse is an asynchronous, event-driven, open-source Security Information and Event Management (SIEM-lite) system designed for modern infrastructure, edge nodes, and security analysts. It unifies real-time authentication log parsing, cryptographic File Integrity Monitoring (FIM), external honeypot telemetry ingestion, IP threat intelligence enrichment, and live tactical dashboard streaming into an accessible, developer-friendly architecture.

🌐 The Real-World Problem Solved

The Pain Points in Modern Security Operations

Challenge	Real-World Impact	How DefendPulse Solves It
Prohibitive SIEM Cost & Complexity	Enterprise SIEMs (Splunk, QRadar, Datadog) cost thousands of dollars per gigabyte of log ingestion and require dedicated engineering teams to maintain. Small-to-medium businesses and dev teams are priced out.	Zero-Cost, Self-Contained Core: DefendPulse delivers turnkey log aggregation, correlation, and live visualization on standard servers with no recurring ingestion fees.
Silent Credential Stuffing & Brute-Force	Web portals and SSH services receive thousands of automated dictionary attacks daily. Traditional servers write these to disk files (`/var/log/auth.log`), where they sit unread until an intrusion has already succeeded.	Sub-Second Streaming Detection: Sliding-window analytical algorithms detect brute-force patterns across multiple sources in under a second and push alerts to operators via WebSockets.
Undetected File Tampering & Web Shells	Attackers exploiting web vulnerabilities alter configuration files, inject backdoors, or tamper with system baselines without leaving obvious traces in standard application logs.	Cryptographic SHA-256 FIM: Continuous baseline integrity scanning flags unauthorized file modifications, file deletions, and unmonitored drop-in files instantly.
Alert Fatigue & Lack of Context	Raw log alerts with just an IP address force security engineers to spend hours querying Whois, geolocation databases, and threat reputation lists manually.	Automated Threat Intelligence: Automatically enriches every offending IP with country of origin, ISP/carrier, AbuseIPDB confidence score, and MITRE ATT&CK technique IDs.
Distributed Attack Visibility Gap	Security teams lack visibility when attacks target decoupled internal or customer-facing microservices.	Non-Blocking Honeypot Telemetry: Provides an authenticated ingestion API that lets satellite websites (`TargetSite`) forward login attempts asynchronously without impacting user experience.

🏗️ High-Level Architecture

DefendPulse combines synchronous web operations with asynchronous network ingestion, decoupled queue workers, and background telemetry tasks:

graph TD
    subgraph External Actors & Ingestion
        HACKER[Attacker / Kali Linux Hydra]
        SYSLOG_DEV[Firewall / Linux Server / Router]
        TARGET_SITE[TargetSite Honeypot :5050]
    end

    subgraph DefendPulse Ingress & Collector Layer
        UDP_PORT[UDP Socket Port 5514]
        UDP_QUEUE[Thread-Safe In-Memory Queue]
        QUEUE_WORKER[Async Syslog Ingest Worker]
        API_INGEST[POST /api/v1/ingest/login-event]
    end

    subgraph Security Analysis & Threat Intelligence
        INTEL[Threat Intel Engine<br/>IP Geolocation, ISP, AbuseIPDB]
        ANALYZER[Log Correlation Engine<br/>Sliding Window Brute-Force Detection]
        FIM_ENGINE[File Integrity Monitor<br/>SHA-256 Baseline Hashes]
    end

    subgraph Persistence & State
        DB[(PostgreSQL / SQLite Database)]
        REDIS[(Redis Message Broker & Cache)]
    end

    subgraph Asynchronous Workers
        CELERY[Celery Analytics Worker]
        BEAT[Celery Periodic Task Scheduler]
    end

    subgraph Live Operator Interface
        SOCKET[Flask-SocketIO / WebSockets]
        UI[Tactical CRT Cyber Dashboard :5000]
        PROM[Prometheus Scraper /metrics]
    end

    subgraph Outbound Notification Channels
        NOTIFY[Alert Router & Throttler]
        EMAIL[SMTP Email Alerts]
        SLACK[Slack Incoming Webhooks]
    end

    %% Flow connections
    HACKER -- Attacks --> TARGET_SITE
    HACKER -- Attacks Port 5514 --> UDP_PORT
    SYSLOG_DEV -- Syslog RFC 3164 --> UDP_PORT
    TARGET_SITE -- HTTP POST + X-API-Key --> API_INGEST

    UDP_PORT --> UDP_QUEUE
    UDP_QUEUE --> QUEUE_WORKER
    QUEUE_WORKER --> INTEL
    API_INGEST --> INTEL
    INTEL --> DB

    DB --> ANALYZER
    ANALYZER --> DB
    FIM_ENGINE --> DB

    BEAT --> REDIS
    REDIS --> CELERY
    CELERY --> ANALYZER
    CELERY --> FIM_ENGINE

    DB --> SOCKET
    SOCKET -- Real-Time Event Push --> UI
    ANALYZER -- On New Threat --> NOTIFY
    NOTIFY -- 10-Min Cooldown --> EMAIL
    NOTIFY -- 10-Min Cooldown --> SLACK
    PROM -- Polls Metrics --> UI

💻 Complete Technology Stack

Technology	Category	Role in DefendPulse
Python 3.10+	Core Language	Runtime language powering the entire backend, correlation engines, and workers.
Flask 3.0+	Web Framework	Exposes HTTP routes, management consoles, REST APIs, and authentication gates.
Flask-SocketIO & Eventlet	Real-Time Engine	Manages high-concurrency bi-directional WebSocket channels for live alert pushes.
SQLAlchemy & Flask-SQLAlchemy	ORM Layer	Manages relational database mapping, indexed time-series queries, and schema abstractions.
Flask-Migrate (Alembic)	Database Migrations	Tracks and executes safe, zero-downtime schema evolution between SQLite and PostgreSQL.
PostgreSQL / SQLite	Database Storage	SQLite for single-file, zero-dependency local setups; PostgreSQL for concurrent production.
Celery & Redis	Task Queue & Broker	Coordinates periodic analytics scans and decouples heavy tasks from web request threads.
Flask-Login	Session Management	Handles operator authentication (AuthN) and Role-Based Access Control (AuthZ).
Flask-Talisman & Flask-Limiter	Defense-in-Depth	Enforces HSTS, CSP, X-Frame-Options headers, and protects against rate-based abuse.
Bootstrap 5 & Chart.js	Frontend UI	Powers the responsive, cyber-themed dark tactical dashboard and live threat ring charts.
Prometheus Client Library	Observability	Exports application telemetry, request latencies, and alert metrics at `/metrics`.
Sentry SDK	Error Tracking	Captures unhandled runtime exceptions with contextual diagnostic telemetry.
MITRE ATT&CK Framework	Threat Taxonomy	Standardizes threat classifications (e.g., T1110 for Brute Force, T1056 for Credential Access).

🛡️ Core Modules & Capabilities

1. High-Performance Network Syslog Listener (`modules/syslog_listener.py`)

Standard RFC 3164 Compliance: Ingests logs over UDP Port 5514 from standard Linux routers, firewalls, and servers.
Non-Blocking Queue Worker Pattern: Uses a decoupled, thread-safe queue (queue.Queue) and a 1 MB socket buffer. The socket thread accepts UDP packets in microseconds, preventing OS-level kernel buffer drops during rapid multi-packet attacks.
Automatic Geolocation & Parsing: Regex-extracts timestamps, process tags, usernames, and IP addresses, immediately querying threat intelligence providers.

2. File Integrity Monitor (FIM) (`modules/file_monitor.py`)

SHA-256 Baseline Hashing: Generates and stores cryptographic checksums of critical configuration files.
State Change Detection: Automatically flags files as OK, TAMPERED (checksum mismatch), or MISSING (file removed), triggering immediate critical alerts.
Untracked File Discovery: Scans monitored directories for unauthorized new files (e.g., uploaded web shells or dropper scripts).

3. Threat Intelligence & Enrichment Engine (`modules/threat_intel.py`)

Automated IP Geolocation: Queries external registries to determine origin country, region, and ISP.
Reputation & Abuse Scoring: Connects to AbuseIPDB to assign confidence-of-abuse scores (0–100%).
MITRE ATT&CK Mapping: Maps detected vectors directly to enterprise MITRE techniques (e.g., T1110 for Brute Force).
Impossible Travel Heuristics: Flags consecutive logins for the same user from geographically incompatible locations within a narrow time window.

4. Alert Routing & Fatigue Prevention (`modules/notifier.py`)

Multi-Channel Dispatch: Sends alerts to operator inboxes via HTML Email (SMTP SSL/TLS) and Slack Incoming Webhooks.
10-Minute Cooldown Throttling: Enforces a per-source, per-vector cooldown window. If an attacker fires 10,000 attempts in an hour, operators receive an initial immediate alert without their inbox being spammed.

5. Production Observability & Health (`/metrics` and `/healthz`)

Prometheus Metrics: Exposes HTTP request counts, query performance, and active alert counters.
Structured JSON Logging: Dumps machine-readable logs ({"timestamp": ..., "level": "INFO", "message": ...}) for ingestion by Fluentd, Vector, or CloudWatch.
Health Check Probe: Returns HTTP 200 with database health status for load balancers and container orchestrators.

🎯 The TargetSite Honeypot System

DefendPulse includes a companion web application called TargetSite (located in /TargetSite) designed specifically for attack and defense simulations.

How TargetSite Integrates with DefendPulse

sequenceDiagram
    autonumber
    actor Attacker as Attacker (Hydra / Kali Linux)
    participant Target as TargetSite (:5050)
    participant Worker as Background Daemon Thread
    participant DefendPulse as DefendPulse SIEM (:5000)
    actor Analyst as Security Operator

    Attacker->>Target: POST /login (username="employee", password="wrong")
    Target->>Target: Check credentials (FAILED)
    Target->>Target: Append to local auth.log
    Target->>Worker: Spawn non-blocking background telemetry thread
    Target-->>Attacker: HTTP 200 ("Invalid credentials") - Instant Response

    Note over Worker,DefendPulse: TargetSite never blocks or crashes even if DefendPulse is offline
    Worker->>DefendPulse: POST /api/v1/ingest/login-event (Header: X-API-Key)
    DefendPulse->>DefendPulse: Validate API Key & Insert LogEntry (source="TargetSite")
    DefendPulse->>DefendPulse: Correlation Engine: 5 failed attempts in 10 mins?
    DefendPulse->>DefendPulse: Raise Alert: "Brute-Force" (Source: TargetSite)
    DefendPulse-->>Analyst: WebSocket Push -> Red Alert + "Source: TargetSite" Badge

Zero Latency Impact: TargetSite uses background daemon threads with a strict 1-second timeout. Even if DefendPulse is offline, TargetSite continues functioning seamlessly.
Authenticated Ingestion: DefendPulse's /api/v1/ingest/login-event endpoint validates incoming events using a shared API key passed in the X-API-Key header.
Visual Source Badging: Alerts originating from TargetSite display a distinctive Source: TargetSite badge on the DefendPulse dashboard.

🚀 Installation & Setup

Prerequisites

Python 3.10+
Git
(Optional) Docker & Docker Compose

Step 1: Clone the Repository

git clone https://github.com/StevinJoseph22/DefendPulse.git
cd DefendPulse

Step 2: Set Up DefendPulse (The Monitor)

Create and Activate Virtual Environment:

# Windows (PowerShell)
python -m venv .venv
.venv\Scripts\activate

# Linux / macOS
python3 -m venv .venv
source .venv/bin/activate

Install Dependencies:
```
pip install -r requirements.txt
```

Configure Environment Variables:

# Copy the example environment file
cp .env.example .env

Edit .env to set your desired shared API key:

SECRET_KEY=your-secure-random-secret-key-here
DATABASE_URL=sqlite:///data/defendpulse.db
DEFENDPULSE_API_KEY=dp_secureshare_9923847291a

Initialize Database Tables & Seed Operator Account:

# Apply database migrations
flask db upgrade

# Seed the default admin operator
python create_admin.py admin_ops admin admin

Start DefendPulse:
```
python app.py
```
DefendPulse is now running at http://127.0.0.1:5000 with the Syslog listener active on UDP port 5514.

Step 3: Set Up TargetSite (The Attack Target)

Open a second terminal window:

Navigate to TargetSite:
```
cd TargetSite
```

Create and Activate Virtual Environment:

# Windows (PowerShell)
python -m venv .venv
.venv\Scripts\activate

# Linux / macOS
python3 -m venv .venv
source .venv/bin/activate

Install TargetSite Dependencies:
```
pip install -r requirements.txt
```

Configure Environment Variables:

cp .env.example .env

Ensure DEFENDPULSE_API_KEY matches the key configured in DefendPulse's .env:

DEFENDPULSE_INGEST_URL=http://127.0.0.1:5000/api/v1/ingest/login-event
DEFENDPULSE_API_KEY=dp_secureshare_9923847291a

Start TargetSite:
```
python app.py
```
TargetSite is now active on http://127.0.0.1:5050.

🧪 Simulating Real-World Attacks

Simulation A: Kali Linux Hydra Attack on TargetSite

From your Kali Linux VM (or any terminal with hydra installed), launch a dictionary brute-force attack against TargetSite:

hydra -l employee -P /usr/share/wordlists/rockyou.txt 127.0.0.1 -s 5050 http-post-form "/login:username=^USER^&password=^PASS^:F=Invalid credentials"

What happens:
1. Hydra sends rapid login attempts to TargetSite on port 5050.
2. TargetSite logs each attempt to TargetSite/logs/auth.log and fires asynchronous telemetry events to DefendPulse.
3. DefendPulse correlates the failed logins, detects the brute-force pattern, flags the attacker's IP, and pushes a glowing red alert with the Source: TargetSite badge to your dashboard!

Simulation B: Network Syslog Attack Simulator

Test the raw UDP Syslog listener on port 5514 using the built-in simulator:

.venv\Scripts\python send_test_syslog.py

What happens: Dispatches 5 simulated failed SSH login packets spoofing remote IP 198.51.100.80. The UDP queue worker parses the events without packet loss and triggers a Brute-Force alert with enriched geolocation metadata.

Simulation C: Local Sandbox Attack Simulator

To test DefendPulse's internal honeypot route (/demo/login):

.venv\Scripts\python attack_simulator.py

What happens: Generates 20 failed login attempts in quick succession to test endpoint rate-limiting and sliding-window alert triggers.

Simulation D: File Tampering & FIM Detection

Log in to DefendPulse at http://127.0.0.1:5000/login.
Navigate to File Integrity Monitor and click Re-calculate & Save Baseline.
Open any monitored file in monitored_files/sample_config.txt and change a character.
Within seconds, the dashboard alerts that the file has been TAMPERED with a cryptographic hash mismatch!

📡 REST API & Observability Endpoints

All REST APIs require the X-API-KEY header set to an authorized operator's key.

Method	Route	Description	Auth Required
`GET`	`/healthz`	Container & load balancer liveness health check.	None
`GET`	`/metrics`	Prometheus-format application metrics export.	None (Rate-limited)
`POST`	`/api/v1/ingest/login-event`	Ingests authentication logs from TargetSite and external apps.	`X-API-Key`
`GET`	`/api/v1/alerts`	Returns JSON array of all active security threats.	`X-API-KEY`
`GET`	`/api/v1/logs`	Returns recent ingested log entries.	`X-API-KEY`
`POST`	`/alerts/resolve/<id>`	Resolves an active security threat alert.	Admin Session

🔑 Default Credentials & Ports Reference

Ports Map

Port	Protocol	Service
5000	HTTP / TCP	DefendPulse Main Web Console & API
5050	HTTP / TCP	TargetSite Demo Employee Portal
5514	Syslog / UDP	DefendPulse Network Syslog Ingestion Listener
6379	TCP	Redis Broker (when running with Celery/Docker)

Default Credentials

Application	URL	Username	Password	Role
DefendPulse Console	`http://127.0.0.1:5000/login`	`admin_ops`	`admin`	Administrator
TargetSite Portal	`http://127.0.0.1:5050/login`	`employee`	`Password123`	Target Employee
DefendPulse Sandbox	`http://127.0.0.1:5000/demo/login`	`admin`	`admin`	Simulated Honeypot

🧪 Automated Testing Suite

DefendPulse includes automated unit and integration tests covering regex parsing, threat calculation, and route authentication:

# Run the complete test suite
python -m pytest

# Run with verbose output
python -m pytest -v

🐳 Docker Deployment

To launch the complete production stack (DefendPulse + PostgreSQL + Redis + Celery):

# Build and start all services in detached mode
docker-compose up --build -d

# Inspect live container logs
docker-compose logs -f

📄 License & Security Disclaimer

License

This project is licensed under the MIT License. See the LICENSE file for details.

Security Disclaimer

DefendPulse and TargetSite are designed for educational, research, and defensive operational monitoring purposes. Always obtain explicit authorization before executing penetration tests or attack simulations against any system or network.

Name		Name	Last commit message	Last commit date
TargetSite		TargetSite
ci/workflows		ci/workflows
data		data
docs		docs
migrations		migrations
modules		modules
monitored_files		monitored_files
static		static
templates		templates
tests		tests
.dockerignore		.dockerignore
.env.example		.env.example
.gitignore		.gitignore
Dockerfile		Dockerfile
LICENSE		LICENSE
README.md		README.md
app.py		app.py
attack_simulator.py		attack_simulator.py
celery_worker.py		celery_worker.py
config.py		config.py
create_admin.py		create_admin.py
docker-compose.yml		docker-compose.yml
models.py		models.py
pytest.ini		pytest.ini
requirements.txt		requirements.txt
send_test_syslog.py		send_test_syslog.py

Latest commit

History

Folders and files

Repository files navigation