ββββββββ ββββββββββ βββ βββββββ βββββββ βββββββ βββββββ ββββββββββββββββββββ βββ
βββββββββββββββββββ ββββββββββββ ββββββββββββββββββββββββββββββββββββββββββββββββββ
ββββββ βββ βββββββββββ βββ βββ βββ βββββββββββ βββ ββββββ ββββββ
ββββββ βββ βββββββββββ βββ βββ βββ βββββββββββ βββ ββββββ ββββββ
βββββββββββββββββββ ββββββββββββ ββββββββββββββββββββ βββ βββ ββββββββββββ βββ
ββββββββ ββββββββββ βββ βββββββ βββββββ ββββββββββ βββ βββ βββββββββββ βββ
Platform Tagline: Institutional-grade Decentralized Intelligence Platform and Automated Second Brain designed for long-term organizational endurance, running on a resilient hybrid storage fabric.
The core inspiration for Echo Cortex emerged during a period of rapid organizational expansion. As teams transitioned to permanent remote setups across multiple global regions, we noticed a critical vulnerability in how organizations store knowledge: conversational leakage.
Countless alignment syncs, sprint planning, and design reviews were held daily, yet the operational insights, assignments, and critical decisions generated inside those sessions dissolved the moment the call ended. Traditional methods, such as manual meeting notes, summaries, or wiki updates, were too tedious to sustain and prone to human bias.
We realized that verbal sessions are high-velocity data assets. Our goal was to design an automated pipeline that intercepts these transient conversational flows and structures them into an immutable, structured relational knowledge graph.
Our engineering team combined expertise in UI/UX, database optimization, and machine learning to build Echo Cortex:
- ABDUL RAZEEK (Lead System Architect): Designed the hybrid storage architecture. Developed fallback paths and relational schemas in MongoDB Atlas and PostgreSQL to prevent data loss.
- GODFREY (Lead Frontend Engineer): Established the Obsidian Stark design system. Programmed the 3D neural lattices using Three.js and built responsive, high-performance dashboard layouts.
- HARIHAR (AI Pipeline & Relational Engineer): Orchestrated the multi-stage machine learning pipeline. Handled Whisper speech-to-text, Sentence Transformers vector mapping, and Gemini contextual recall.
- PRITHIVIRAAJJ (Security & Governance Specialist): Implemented cryptographic logging, local isolation profiles, role-based authorization middleware, and user privacy compliance tools.
We faced a significant challenge during initial development: modern SaaS applications rely heavily on high-availability web APIs. If the cloud database cluster experiences latency, or if a user is operating in a network-restricted local environment, typical transcription and logging systems fail completely.
Our goal was to design a platform that remains operational even when the primary database cluster is unreachable. This required building a local-first memory fallback system, database-independent configurations, and self-repairing vector maps that can re-sync when network connections restore.
Echo Cortex is designed around the Obsidian Stark design language, which prioritizes data integrity and visual clarity through high-contrast, distraction-free layouts:
- Obsidian Foundation: We chose a deep black background (
#050505) to reduce eye strain and focus the user's attention on transcript data, dashboards, and charts. - Stark Typography: The interface pairs
Interfor general UI text withJetBrains Monofor granular logs, code blocks, and configuration properties. - Signal Elements: We used neon green accents (
#00FF41) to represent system health, alongside deep violet markers for active focus areas in transcripts.
We began by coding the Express backend and testing the initial transcription pipeline. Early versions relied on sequential cloud API requests, which introduced noticeable processing latency.
To address this, we reorganized the pipeline into parallel execution tracks, allowing speech-to-text conversion and feature vector extraction to run concurrently. We then integrated the Google Gemini SDK for conversational retrieval and built a custom React SPA layout utilizing decentralized import maps to speed up load times.
Because voice sessions contain sensitive institutional data, security was integrated directly into the system's core:
- Cryptographic Audit Trails: The pipeline writes a SHA-256 hash of each ingested transcript to a compliance ledger, verifying data integrity.
- Scoped Authorization: Backend routes are protected by middleware that checks user roles and API keys before allowing data access.
- API Key Isolation: All external API calls (Hugging Face, Groq, Gemini) are handled server-side using secure environment variables, keeping API keys hidden from the client browser.
To comply with global data protection standards, Echo Cortex enforces strict consent controls:
- Explicit Recording Permissions: The user interface prompts for explicit recording approval before accessing the browser's
MediaRecorderAPI. - Content Redaction: The system allows users to flag speakers or mask specific terms within transcripts before committing data to vector indexes.
- Data Deletion Compliance: Triggering the delete route purges the database entry and deletes all related vector embeddings and cached transcripts.
We designed the application around a single-page view with a collapsible sidebar, avoiding deep, nested menus. This layout lets users record sessions, review transcripts, inspect action items, and trigger comparisons in a unified workspace. We also added a 3D neural lattice backdrop via Three.js to represent the underlying vector database space.
- Handling Ingestion CastErrors: When MongoDB went offline, Mongoose threw
CastErrorexceptions when parsing custom local IDs (e.g.,m-17189234857).- Solution: We added explicit
mongoose.Types.ObjectId.isValid(id)checks to bypass MongoDB lookups and search local memory when a non-standard ID format is detected.
- Solution: We added explicit
- Whisper API Timeout Mitigation: Sending large base64-encoded audio files directly via REST APIs occasionally caused socket timeouts.
- Solution: We configured buffered chunking and added a structured fallback script to return default data during API warm-up periods.
Our team used an API-first development process. We defined the TS models in types/meeting.ts before writing any frontend or backend code. This decoupled approach allowed the frontend team to build layouts with mock data structures while the backend team worked on database schemas, controllers, and pipeline logic.
We learned that AI systems must be designed for network resilience. Relying solely on cloud databases introduces vulnerability to external outages. By implementing local-first memory fallbacks and database-independent sync logic, we ensured that users can capture and store critical meeting records under any network conditions.
Our plans for Echo Cortex include deploying local offline models (such as Llama.cpp and local ONNX runtimes), adding multi-speaker diarization, and integrating graph databases (such as Neo4j) to support deeper relationship mapping.
The name Echo Cortex represents the dual nature of our platform:
- Echo: The acoustic signature of conversations and meetings captured by the system.
- Cortex: The neural processing layer that organizes raw audio into structured, relational intelligence.
"We built Echo Cortex to solve our own team alignment challenges. We hope this platform helps your team secure knowledge, capture alignment, and maintain absolute structural clarity."
- Core Concept
- Platform Modules
- System Architecture Overview
- Tech Stack & Core Dependencies
- Detailed Folder Structure
- UI Component Documentation
- End-to-End Ingestion Pipeline
- API Reference Specifications
- Database Resiliency Policy & Schemas
- Setup, Configuration & Installation
- Testing & Verification Guide
- Scalability & Security Considerations
- Obsidian Stark Design Tokens
- Frequently Asked Questions (FAQ)
- Contributors & Licensing
In modern organizations, transient conversational assets are lost immediately after a meeting ends. Echo Cortex treats conversations as structured data. The system goes beyond basic linear transcription by extracting:
[Spoken Session] ββ> [Whisper STT] ββ> [Semantic Vectorizer] ββ> [Groq LLM Parser] ββ> [Structured Graph]
βββ Chronological Segments
βββ Action Commitments
βββ Strategic Decisions
- Segmented Dialogue: Timestamps mapped to validated speaker nodes.
- Action Commitments: Clear, owner-linked tasks with progress tracking.
- Strategic Decisions: Verifiable outcomes with confidence scores and context links.
- Semantic Vector Space: A 384-dimensional vector database for context retrieval.
Echo Cortex features five modules designed to support institutional intelligence and data governance:
| Module | Purpose | Visualizations |
|---|---|---|
| Strategic Analytics | Global system health & metrics | KPI Sparklines, Ribbon Chart, Goal Tree, Word Cloud |
| Delta Audit | Meeting-to-meeting comparison | Clustered Columns, Slope Chart, Target Bullet Charts |
| Entity Graph | Network relation mapping | HTML5 Canvas Navigator, Sankey Charts, Expertise Heatmap |
| Compliance Vault | Data retention & auditing | Gantt Charts, Waterfall Charts, Telemetry Gauges, Ledger |
| Synapse Hub | External system integration | Jira Integrations, Slack webhooks, Support node routes |
Provides high-level visibility into institutional data velocity and system performance.
- KPI Sparkline Cards: Custom SVG line graphs displaying Institutional IQ and Knowledge Velocity trends.
- Departmental Ribbon Chart: Visualizes Knowledge Velocity rankings across organizational divisions over time.
- Goal Decomposition Tree: A collapsible tree diagram drilling down from corporate objectives into active team sessions.
- Transcript Word Cloud: An interactive keywords grid featuring scale-on-hover micro-animations.
- Operational Onboarding Handbook: Step-by-step dashboard cards detailing audio ingestion, STT, and Compliance Vault retention.
- Split-Screen Login: An authentication panel featuring a neural lattice animation, clock widget, and login/registration inputs.
- Local Branding Assets: Served via favicon.png and logo.png.
Compares two ingested sessions side-by-side to track action item completion and changes in decisions.
- Clustered Column Chart: Displays comparative metrics for action items and decision confidence levels.
- Decision Slope Chart: Maps the evolution of decisions from tentative to finalized states.
- Target Progress Bullet Chart: Measures current execution progress against historical baselines.
- AI Smart Narrative: Renders auto-generated comparative summaries from transcript content.
Maps connections, speaker participation profiles, and skills across teams.
- Network Navigator Canvas: An interactive HTML5 Canvas that highlights speakers and connected team nodes.
- Information Flow Sankey Chart: Visualizes the pipeline path from raw audio segments to final decisions.
- Expertise Mapping Heatmap: Cell grids mapping team members to specific skills with high-contrast background gradients.
Governs the lifecycle, audit trails, and pipelines of ingested assets.
- Artifact Lifecycle Gantt Chart: Tracks retention stages (Created -> Audited -> Archived) of meeting assets.
- Decision Ingest Waterfall Chart: Illustrates base segments adjusted by action items to reach final Immutable Logs.
- Telemetry Gauges: Radial dial indicators monitoring loads on external AI API channels.
- AI Morality Alignment Principles: Hardcoded compliance controls covering human-in-the-loop validation, fairness, and right-to-forget protocols.
- Immutable Cryptographic Ledger: A transparent table tracking operations and SHA-256 origin proofs.
Manages integrations and system support:
- Atlassian Jira Issues: Synchronizes development action items to task backlogs.
- Slack Channels: Streams decisions to target communication feeds.
- Support Node: Direct links targeting administrative support with hover animations.
Echo Cortex divides components into ingestion, storage, and recall layers:
graph TD
A[Client UI: React 19] -->|Base64 Audio / Text Report| B[Express API Gateway: Port 2348]
B -->|Pipeline Trigger| C[Echo Cortex Pipeline]
subgraph Ingestion Pipeline
C -->|1. Raw Audio WebM| D[Hugging Face: Whisper Large v3 STT]
D -->|2. Refined Transcript| E[Hugging Face: Sentence Transformers Embedding]
C -->|3. Transcript + Prompt| F[Groq LLM: Llama-3.1-8b-instant Analysis]
end
subgraph Storage Layer
B -->|Mongoose Handshake| G[(MongoDB Atlas Cluster)]
B -->|PostgreSQL Pool| H[(Supabase PostgreSQL Database)]
G -.->|Fallback if Offline| I[[Local In-Memory Storage]]
end
subgraph Recall Layer
A -->|Chat Query / Vector Query| J[Gemini 3 Flash LLM]
J -->|Contextual Response| A
end
sequenceDiagram
participant User as Client UI
participant Server as Express Server
participant HF as Hugging Face APIs
participant Groq as Groq Ingestion Layer
participant DB as Hybrid Storage
User->>Server: POST /api/meetings/ingest (audio/report)
alt Audio Payload Provided
Server->>HF: Audio Buffer -> Whisper Large v3
HF-->>Server: Return Plaintext Transcript
else Plaintext Report Provided
Note over Server: Bypasses Speech-to-Text Ingestion
end
Server->>HF: inputs: Transcript -> Sentence Transformers
HF-->>Server: Return 384-Dim Vector Embeddings
Server->>Groq: JSON Schema Prompt + Transcript
Groq-->>Server: Return Structured JSON (segments, actions, decisions)
Server->>HF: inputs: Summary Text -> MMS-TTS
HF-->>Server: Return Speech Synthesis Waveform (Base64)
alt Database Cluster Connected
Server->>DB: Write MongoDB Meeting & Postgres relational tables
else Offline state
Server->>Server: Push to local memory storage array
end
Server-->>User: HTTP 201 Created (artifactId + structured analysis)
stateDiagram-v2
[*] --> Initialize
Initialize --> ConnectMongo
ConnectMongo --> MongoActive : Connection Succeeded
ConnectMongo --> MongoOffline : Connection Timeout
state MongoActive {
[*] --> IngestDataMongo
IngestDataMongo --> WriteToMongoDB
WriteToMongoDB --> [*]
}
state MongoOffline {
[*] --> IngestDataMemory
IngestDataMemory --> CheckIdFormat
CheckIdFormat --> SearchLocalMemory : Id begins with 'm-'
CheckIdFormat --> RaiseError : Invalid ObjectId syntax
SearchLocalMemory --> [*]
}
MongoOffline --> ConnectMongo : Retry Interval (30s)
Echo Cortex organizes its dependencies into logical application layers:
| Component | Library/Framework | Description |
|---|---|---|
| Core View Framework | React 19 (ESM native) | Render engine using hooks, contexts, and React.Suspense layers. |
| 3D Rendering | Three.js | Powers the interactive, animated neural network mesh backdrop. |
| Animation UI | Framer Motion & GSAP | Controls transitions, workspace panels, and metric displays. |
| Style Tokens | Vanilla CSS | Custom styling system using the stark high-contrast design theme. |
| Icons | Lucide React | Provides vector icons for UI navigation and status indicators. |
| Application State | React context hooks | Handles global data flow (AuthContext & TeamContext). |
- react & react-dom (v19.0.0): Uses React 19's concurrent rendering features, compiler-optimized updates, and native support for async actions during forms submissions.
- three (v0.184.0): Manages canvas initialization, light shaders, and custom particle configurations for the animated 3D network backdrop.
- framer-motion (v12.38.0): Coordinates panel slide-outs, tab switches, and hover scaling for key dashboard components.
- lucide-react (v0.563.0): Custom SVG icons used to indicate system state across the interface.
| Component | Library/Framework | Description |
|---|---|---|
| Application Runtime | Node.js (v18+) | Server-side runtime executing TypeScript configurations. |
| REST Router | Express.js | Exposes endpoints for data ingestion, search, and configuration. |
| Authentication | Firebase Admin SDK | Manages authorization profiles and token validation. |
| LLM Orchestrator | Google GenAI SDK | Powers conversational retrieval and query processing using Gemini 3. |
| Live Transpiler | ts-node-dev | Supports hot-reloading during backend development. |
| Component | Library/Framework | Description |
|---|---|---|
| NoSQL Storage | MongoDB Atlas | Stores meeting data, segments, action items, and transcripts. |
| Object Data Mapper | Mongoose v8.x | Manages Mongo schemas, validation checks, and query patterns. |
| Relational Database | Supabase Postgres | Manages relational tables and configuration parameters. |
| Vector Search | pgvector extension | Computes cosine distance matches for 384-dimensional embeddings. |
| Local Cache | In-Memory Array | Temporarily stores data when external database clusters are offline. |
The project layout separates frontend visual components from the backend API server logic:
echo/
βββ App.tsx # React root component coordinating views and providers
βββ index.html # HTML entry point (Tailwind config & ESM maps)
βββ index.css # Custom Vanilla CSS defining Obsidian Stark variables
βββ package.json # Project manifest and dev dependencies
βββ tsconfig.json # TypeScript compiler configuration
βββ vite.config.ts # Vite configurations for build and routing
βββ launch_echo.bat # Batch script to launch backend & frontend concurrently
βββ logo.png # Brand asset
βββ backend/ # Core Express API Application
β βββ index.ts # Application bootstrapper
β βββ package.json # Backend dependencies
β βββ tsconfig.json # Backend compilation targets
β βββ src/ # API Source Code
β βββ bootstrap/ # Startup processes
β βββ config/ # Database & AI credentials setup
β β βββ database.ts # MongoDB cluster status checker
β β βββ firebase.ts # Firebase administrative configurations
β β βββ gemini.ts # Gemini client setup
β βββ controllers/ # Express HTTP Route Controllers
β β βββ auth.controller.ts
β β βββ config.controller.ts
β β βββ logs.controller.ts
β β βββ meetings.controller.ts
β βββ jobs/ # Asynchronous queue workers
β βββ middleware/ # Request validation layers
β β βββ auth.middleware.ts
β βββ models/ # Mongoose MongoDB models
β β βββ meeting.model.ts
β β βββ user.model.ts
β βββ routes/ # API Endpoints Route declarations
β β βββ admin.routes.ts
β β βββ auth.routes.ts
β β βββ config.routes.ts
β β βββ logs.routes.ts
β β βββ meetings.routes.ts
β β βββ search.routes.ts
β β βββ transcript.routes.ts
β βββ services/ # Integration business logic
β βββ types/ # TS interface structures
β βββ utils/ # General helper modules
βββ components/ # Frontend React components
β βββ Recorder.tsx # Main recording overlay
β βββ Timeline.tsx # Horizontal timeline view
β βββ audio/ # Ingestion audio interfaces
β β βββ AudioPlayer.tsx
β β βββ AudioRecorder.tsx
β β βββ Waveform.tsx
β βββ chat/ # Contextual chat components
β β βββ ChatWithNotes.tsx
β βββ landing/ # Public page layout modules
β βββ layout/ # Structural navigation shells
β β βββ Sidebar.tsx
β βββ search/ # Advanced semantic search panels
β βββ ui/ # Shared UI components
β βββ AICompanion.tsx
β βββ Badge.tsx
β βββ Button.tsx
β βββ Input.tsx
β βββ Loader.tsx
β βββ Scene3D.tsx
βββ context/ # React State Contexts
β βββ AuthContext.tsx
β βββ TeamContext.tsx
β βββ ThemeContext.tsx
βββ pages/ # Route Page Views
β βββ Admin.tsx # System and file-tree settings
β βββ ComplianceVault.tsx # Retention settings
β βββ CreateMeeting.tsx # Recording dashboard
β βββ Dashboard.tsx # Global workspace dashboard
β βββ DeltaAudit.tsx # Dual meeting audits
β βββ EntityGraph.tsx # Node visualization panels
β βββ MeetingDetail.tsx # Detail view with chat
βββ services/ # Frontend integrations
β βββ firebase.ts
β βββ geminiService.ts
βββ types/ # Frontend types
βββ meeting.ts
- App.tsx: Coordinates application routing and layouts. It imports page modules (such as
DeltaAudit,EntityGraph, andComplianceVault) and passes data throughAuthProvider,TeamProvider, andThemeProvider. - index.html: Defines the web interface metadata, imports design fonts (Inter & JetBrains Mono), loads Tailwind definitions, and declares the ES import maps.
- index.css: Sets color tokens, typography rules, layout variables, scrollbar structures, and hover animations.
- components/ui/Scene3D.tsx: Renders the interactive 3D particle lattice that updates based on the user's focus and application state.
- components/ui/AICompanion.tsx: Provides an interactive panel for querying meeting transcripts using natural language.
- components/audio/AudioRecorder.tsx: Connects to the browser's audio capture APIs, displays recording state, and sends encoded audio to the ingestion backend.
- pages/Admin.tsx: Displays system configurations, active databases, file directories, and provides a panel for managing user access permissions.
- pages/DeltaAudit.tsx: Compares two transcripts side-by-side, displaying changes in action items and decision metrics.
- pages/EntityGraph.tsx: Renders an interactive relationship graph mapping speakers to meeting segments using HTML5 Canvas.
This section provides technical details for key visual components in Echo Cortex.
Renders the animated particle structure representing the vector space.
- Framework: Three.js integrated via native Canvas frames.
- Architecture: Uses a particle mesh buffer geometry. Nodes are linked dynamically in the animation loop based on distance thresholds.
- Properties:
particleCount: Default value is200.connectionDistance: Renders linking lines for nodes within1.5units.
- Styling: Particle colors blend between neon green (
#00FF41) and deep cortex cyan (#00e5ff).
Handles voice session capture on the client side.
- APIs: Uses the browser's native
MediaRecorderAPI. - Parameters:
mimeType: Defaults toaudio/webm;codecs=opus.sampleRate: Set to48000Hz.
- State Indicators: Displays recording duration, active volume level metrics, and a dynamic waveform display.
Renders the interactive relationship graph.
- Framework: Built using the HTML5 2D Canvas context.
- Physics Engine: Implements a basic force-directed layout that positions nodes representing speakers and meeting segments dynamically.
- Interactions: Supports clicking nodes to focus on specific segments and zooming/panning across the canvas.
- React Props: Accepts
segments: TranscriptSegment[]andonNodeClick: (segmentId: string) => void. - Behavior: Maps speaker nodes along a chronological grid. Highlights key segments containing action items or decisions with neon-bordered containers.
- State Management: Coordinates the chat input and history.
- Behavior: Submits chat requests to
/api/meetings/chatand streams the parsed markdown output from the Gemini LLM.
Ingesting audio or text reports triggers a 4-phase synchronization pipeline:
Ingestion Event Triggered
β
ββββββββββββββββββ΄βββββββββββββββββ
βΌ βΌ
[Audio WebM Buffer] [Text Report Ingestion]
β β
(Phase 1: Whisper STT) β (Bypass STT)
β β
ββββββββββββββββββ¬βββββββββββββββββ
βΌ
(Phase 2: Sentence Vectors)
β
βΌ
(Phase 3: Database Sync)
β
βΌ
(Phase 4: Groq Extract JSON)
β
βΌ
(Phase 5: MMS-TTS Synthesis)
When a user records a session, the frontend captures audio chunks using the browser-native MediaRecorder API (audio/webm at a 48kHz sampling rate) and sends them to the backend as an encoded base64 string. The backend processes the string, converts it to an audio buffer, and submits it to the Hugging Face Whisper API:
- Model Endpoint:
https://api-inference.huggingface.co/models/openai/whisper-large-v3 - Authorization: Uses the
HF_TOKENenvironment variable. - Latency Mitigation: If the external API is cold-starting or rate-limited, the system falls back to a default dialogue transcript to ensure database sync remains active.
// Node.js implementation executing Hugging Face Speech-to-Text request
const audioBuffer = Buffer.from(audioBase64, 'base64');
const sttResponse = await fetch('https://api-inference.huggingface.co/models/openai/whisper-large-v3', {
method: 'POST',
headers: {
'Authorization': `Bearer ${process.env.HF_TOKEN}`,
'Content-Type': mimeType || 'audio/webm'
},
body: audioBuffer
});
const sttData = await sttResponse.json();
const transcriptText = sttData.text;Once transcription completes, the plain text is sent to the Sentence Transformers API to generate semantic search vectors:
- Model Endpoint:
https://api-inference.huggingface.co/pipeline/feature-extraction/sentence-transformers/all-MiniLM-L6-v2 - Output Vector: A 384-dimensional dense array representing the semantic content of the transcript.
- Application: Used for cosine similarity checks and semantic searches across stored sessions.
// Node.js implementation querying Sentence Transformers embeddings
const embedResponse = await fetch('https://api-inference.huggingface.co/pipeline/feature-extraction/sentence-transformers/all-MiniLM-L6-v2', {
method: 'POST',
headers: {
'Authorization': `Bearer ${process.env.HF_TOKEN}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({ inputs: transcriptText })
});
const embedData = await embedResponse.json();
const embeddingVector = Array.isArray(embedData[0]) ? embedData[0] : embedData;The plain text transcript is sent to the Groq API alongside a system prompt enforcing a strict JSON schema:
- Model:
llama-3.1-8b-instant - JSON Schema:
{ "segments": [ { "speaker": "String", "text": "String" } ], "actionItems": [ { "description": "String", "owner": "String" } ], "decisions": [ { "summary": "String" } ] } - Validation: The backend parses the JSON payload. If the API is rate-limited, the system inserts default structured segments to maintain database consistency.
// Node.js implementation executing Groq Structured Ingestion Analysis request
const groqResponse = await fetch('https://api.groq.com/openai/v1/chat/completions', {
method: 'POST',
headers: {
'Authorization': `Bearer ${process.env.GROQ_API_LLM}`,
'Content-Type': 'application/json'
},
body: JSON.stringify({
model: process.env.GROQ_MODEL || 'llama-3.1-8b-instant',
messages: [
{ role: "system", content: "Analyze this transcript. Return ONLY a JSON object strictly following the structure: {\"segments\":[{\"speaker\":\"Name\",\"text\":\"Text\"}],\"actionItems\":[{\"description\":\"Task\",\"owner\":\"Name\"}],\"decisions\":[{\"summary\":\"Decision\"}]}" },
{ role: "user", content: transcriptText }
],
response_format: { type: "json_object" },
temperature: 0.1
})
});
const groqData = await groqResponse.json();
const structuredOutput = JSON.parse(groqData.choices[0].message.content);After generating decisions and action items, the system compiles the primary decision summary into a short text message and sends it to the Facebook MMS-TTS model:
- Model Endpoint:
https://api-inference.huggingface.co/models/facebook/mms-tts-eng - Return Payload: An audio summary encoded in base64, allowing users to listen to the main outcome directly from the dashboard.
Processes audio payloads or text documents through the transcription and extraction pipeline.
- Endpoint:
POST /api/meetings/ingest - Headers:
Content-Type: application/json Authorization: Bearer <ID_TOKEN>
{
"audio": "UklGRigAAABXQVZFZm10IBIAAAABAAEARKwAAIhYAQACABAAAABkYXRhAgAAAAAA...",
"mimeType": "audio/webm"
}{
"report": "Sarah: Let's finalize the database architecture. Dave: I recommend moving to MongoDB Atlas."
}{
"message": "Artifact committed to Echo knowledge graph",
"artifactId": "65b8f2d59dfa3847e12089ba",
"data": {
"segments": [
{ "speaker": "Sarah", "text": "Let's finalize the database architecture." },
{ "speaker": "Dave", "text": "I recommend moving to MongoDB Atlas." }
],
"actionItems": [
{ "description": "Migrate database architecture to MongoDB Atlas", "owner": "Dave" }
],
"decisions": [
{ "summary": "Migrate database architecture to MongoDB Atlas" }
],
"summarySpeechBase64": "UklGRjIAAABXQVZFZm10IBIAAA..."
}
}{
"error": "No audio or report provided"
}Submit natural language queries to retrieve context from a specific meeting transcript.
- Endpoint:
POST /api/meetings/chat - Headers:
Content-Type: application/json Authorization: Bearer <ID_TOKEN>
- Request Body:
{
"meetingId": "65b8f2d59dfa3847e12089ba",
"query": "Who proposed migrating the database?"
}- Response (HTTP 200 OK):
{
"reply": "According to the transcript, Dave recommended migrating to MongoDB Atlas, and the action item was assigned to him."
}Retrieves all meeting documents stored in the database.
- Endpoint:
GET /api/meetings - Headers:
Authorization: Bearer <ID_TOKEN> - Response (HTTP 200 OK):
[
{
"_id": "65b8f2d59dfa3847e12089ba",
"title": "SaaS Architecture Review",
"transcript": "Sarah: Let's finalize the database architecture. Dave: I recommend moving to MongoDB Atlas.",
"embedding": [0.0125, -0.0456, 0.0892, 0.1235],
"segments": [
{ "speaker": "Sarah", "text": "Let's finalize the database architecture." },
{ "speaker": "Dave", "text": "I recommend moving to MongoDB Atlas." }
],
"actionItems": [
{ "description": "Migrate database architecture to MongoDB Atlas", "owner": "Dave" }
],
"decisions": [
{ "summary": "Migrate database architecture to MongoDB Atlas" }
],
"summarySpeechBase64": "UklGRjIAAABXQVZFZm10IBIAAA...",
"createdAt": "2026-06-05T14:50:00.000Z"
}
]Deletes a specific meeting record and all associated data from the storage system.
- Endpoint:
DELETE /api/meetings/:id - Headers:
Authorization: Bearer <ID_TOKEN> - Response (HTTP 204 No Content): No content returned on successful deletion.
Retrieves active configurations required to initialize front-end services.
- Endpoint:
GET /api/config/firebase - Response (HTTP 200 OK):
{
"apiKey": "AIzaSyAYfihj4mgecCfidY1gchpicFf1zkVeM5Y",
"authDomain": "echo-cortex.firebaseapp.com",
"projectId": "echo-cortex",
"storageBucket": "echo-cortex.appspot.com",
"messagingSenderId": "874593847592",
"appId": "1:874593847592:web:aef874aef874"
}Updates configuration parameters in the system database. Requires administrative access.
- Endpoint:
PUT /api/config/firebase - Headers:
Content-Type: application/json Authorization: Bearer <ADMIN_TOKEN>
- Request Body:
{
"apiKey": "AIzaSyAYfihj4mgecCfidY1gchpicFf1zkVeM5Y",
"authDomain": "updated-domain.firebaseapp.com",
"projectId": "echo-cortex-updated",
"storageBucket": "updated-bucket.appspot.com",
"messagingSenderId": "874593847592",
"appId": "1:874593847592:web:aef874aef874",
"active": true
}- Response (HTTP 200 OK):
{
"message": "Firebase configuration updated successfully"
}Receives front-end logs and writes them to the system database for monitoring and debugging.
- Endpoint:
POST /api/logs - Headers:
Content-Type: application/json - Request Body:
{
"level": "error",
"message": "Failed to compile Three.js scene dependencies",
"meta": {
"agent": "Mozilla/5.0",
"viewport": "1920x1080"
}
}- Response (HTTP 200 OK):
{
"status": "logged"
}Echo Cortex implements a dual-tier storage strategy to ensure high availability and prevent data loss during network disruptions.
Ingestion Event Triggered
β
Is MongoDB Active?
ββββββββββββββ΄βββββββββββββ
βΌ YES βΌ NO
[Write to MongoDB] [Write to Local RAM Cache]
β β
β Verify Connection
β ββββββββββ΄βββββββββ
β βΌ Online βΌ Offline
β [Sync to MongoDB] [Retain in RAM]
βΌ β
Sync Complete β
ββββββββββ¬ββββββββ
βΌ
Postgres Sync (relational logs)
- Format Verification: Before querying the MongoDB cluster, the backend validates IDs using
mongoose.Types.ObjectId.isValid(id). - RAM Ingestion Route: If a database connection is unavailable, or if the ID prefix indicates a local record (e.g.,
m-), the system queries the local in-memory storage array (inMemoryMeetings). - Active Sync Reconciliation: When the connection is restored, local in-memory records are merged with MongoDB data. Duplicate primary keys are filtered out to keep client dashboards consistent.
- Error Handling: If both primary and fallback storage tiers are empty, the application returns explicit HTTP errors to the client rather than serving blank mock screens.
import mongoose, { Document, Schema } from 'mongoose';
export interface IMeeting extends Document {
title: string;
transcript: string;
embedding: number[];
segments: any[];
actionItems: any[];
decisions: any[];
summarySpeechBase64?: string;
createdAt: Date;
}
const meetingSchema = new Schema<IMeeting>({
title: { type: String, default: 'SaaS Architecture Review' },
transcript: { type: String, required: true },
embedding: { type: [Number], default: [] },
segments: { type: [Object], default: [] },
actionItems: { type: [Object], default: [] },
decisions: { type: [Object], default: [] },
summarySpeechBase64: { type: String, default: '' },
createdAt: { type: Date, default: Date.now }
});
export const Meeting = mongoose.model<IMeeting>('Meeting', meetingSchema);import mongoose, { Document, Schema } from 'mongoose';
export interface IUser extends Document {
name: string;
email: string;
passwordHash: string;
role: string;
avatar: string;
createdAt: Date;
}
const userSchema = new Schema<IUser>({
name: { type: String, required: true },
email: { type: String, required: true, unique: true },
passwordHash: { type: String, required: true },
role: { type: String, default: 'admin' },
avatar: { type: String, required: true },
createdAt: { type: Date, default: Date.now }
});
export const User = mongoose.model<IUser>('User', userSchema);The Supabase PostgreSQL database schemas track configurations, sessions, and compliance trails:
-- Enable pgvector extension for similarity search
CREATE EXTENSION IF NOT EXISTS vector;
-- Create Core Firebase Config table
CREATE TABLE firebase_config (
id SERIAL PRIMARY KEY,
api_key VARCHAR(500) NOT NULL,
auth_domain VARCHAR(255) NOT NULL,
project_id VARCHAR(255) NOT NULL,
storage_bucket VARCHAR(255) NOT NULL,
messaging_sender_id VARCHAR(255) NOT NULL,
app_id VARCHAR(255) NOT NULL,
active BOOLEAN DEFAULT true NOT NULL,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
);
-- Ingest logs generated during browser sessions
CREATE TABLE client_logs (
id SERIAL PRIMARY KEY,
level VARCHAR(50) NOT NULL,
message TEXT NOT NULL,
meta JSONB DEFAULT '{}'::jsonb,
timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
);
-- Core Meeting Entity representation
CREATE TABLE meetings (
id VARCHAR(255) PRIMARY KEY,
title VARCHAR(500) NOT NULL,
transcript TEXT NOT NULL,
embedding vector(384), -- pgvector extension column for dense semantic vectors
summary_speech_base64 TEXT,
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP NOT NULL
);
-- Transcript Segments mapped to validated speakers
CREATE TABLE segments (
id VARCHAR(255) PRIMARY KEY,
meeting_id VARCHAR(255) REFERENCES meetings(id) ON DELETE CASCADE,
speaker VARCHAR(255) NOT NULL,
text TEXT NOT NULL,
start_time DECIMAL(10,2) DEFAULT 0.00,
end_time DECIMAL(10,2) DEFAULT 0.00
);
-- Action Items extract representation
CREATE TABLE action_items (
id VARCHAR(255) PRIMARY KEY,
meeting_id VARCHAR(255) REFERENCES meetings(id) ON DELETE CASCADE,
description TEXT NOT NULL,
owner VARCHAR(255) DEFAULT 'Unassigned',
status VARCHAR(50) DEFAULT 'pending',
source_segment_id VARCHAR(255) REFERENCES segments(id) ON DELETE SET NULL
);
-- Decisions extracted
CREATE TABLE decisions (
id VARCHAR(255) PRIMARY KEY,
meeting_id VARCHAR(255) REFERENCES meetings(id) ON DELETE CASCADE,
summary TEXT NOT NULL,
confidence_score DECIMAL(3,2) DEFAULT 1.00,
source_segment_id VARCHAR(255) REFERENCES segments(id) ON DELETE SET NULL
);Ensure the following tools are installed on your host system:
- Node.js: Version 18.0.0 or higher.
- NPM: Version 9.0.0 or higher.
- MongoDB Cluster: A running MongoDB Atlas instance or a local MongoDB database.
- Supabase PostgreSQL (Optional): A database instance supporting the
pgvectorextension.
Execute these commands in your terminal to initialize and install project packages:
# Clone the codebase
git clone https://github.com/your-org/echo-cortex.git
cd echo-cortex
# Install frontend and workspace root modules
npm install
# Navigate and install backend components
cd backend
npm install
cd ..Create a .env file in the root workspace directory with these variable definitions:
# Server Ingest API Configuration
PORT=2348
# MongoDB Connections
MONGODB_URI=mongodb+srv://<username>:<password>@cluster0.mongodb.net/echo_db?retryWrites=true&w=majority
# Supabase PostgreSQL Connections
DATABASE_URL=postgresql://postgres:<password>@db.supabase.co:5432/postgres
# Hugging Face AI pipeline Token (Whisper + Sentence Transformers)
HF_TOKEN=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
# Groq LLM API Configurations
GROQ_API_LLM=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
GROQ_MODEL=llama-3.1-8b-instant
# Gemini LLM API Configurations
GEMINI_API_LLM=AIzaSyAYfihj4mgecCfidY1gchpicFf1zkVeM5Y
# Client API Target Router
VITE_API_URL=http://localhost:2348To run Echo Cortex inside containerized environments, create a Dockerfile at the root directory:
# Dockerfile definition
FROM node:18-alpine
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
EXPOSE 2348
CMD ["npm", "run", "dev"]# docker-compose.yml configuration
version: '3.8'
services:
backend:
build:
context: ./backend
dockerfile: Dockerfile
ports:
- "2348:2348"
environment:
- PORT=2348
- MONGODB_URI=mongodb://mongodb:2717/echo_db
- HF_TOKEN=hf_mock_token
depends_on:
- mongodb
mongodb:
image: mongo:latest
ports:
- "2717:27017"
volumes:
- mongo_data:/data/db
volumes:
mongo_data:Build and run the container:
docker build -t echo-cortex .
docker run -p 2348:2348 --env-file .env echo-cortexTo keep the application running continuously in production environments:
# Install PM2 globally
npm install pm2 -g
# Start the Express server with PM2
cd backend
pm2 start ts-node --name "echo-backend" -- src/index.tsTo host the frontend visual applications and route backend API handlers on Vercel, use this vercel.json file configuration:
{
"version": 2,
"rewrites": [
{
"source": "/api/(.*)",
"destination": "http://localhost:2348/api/$1"
},
{
"source": "/(.*)",
"destination": "/index.html"
}
]
}Echo Cortex includes a test script to verify that database connections and model interfaces are configured correctly.
Execute the test script from the root directory:
node test_endpoints.jsThe test script verifies the following features:
- Health Check Validation: Confirms
GET /healthreturns an operational status code. - Authentication Handshake: Submits credentials to
POST /api/auth/loginto confirm token validation. - Meeting List Retrieval: Queries
GET /api/meetingsto verify database readout. - Ingestion Inferences: Submits base64 audio data to
POST /api/meetings/ingestto test the Whisper transcription and Groq schema parser. - Gemini Ingest Chat: Submits a query to
POST /api/meetings/chatto verify document recall. - Vector Score Ranking: Validates similarity checks at the
/api/searchendpoint.
A successful test run will return the following console logs:
[ASSERTION 1] GET /health returning operational server status... PASS
[ASSERTION 2] Resolving authentication handshake... PASS
[ASSERTION 3] Fetching institutional meetings... PASS
[ASSERTION 4] Ingesting mock base64 audio buffers... PASS
[ASSERTION 5] Querying Gemini Conversational Recall... PASS
[ASSERTION 6] Validating vector similarity rankings... PASS
All verification tests completed successfully.
As the vector database grows, searching across thousands of multi-dimensional embeddings can degrade query performance. To maintain search speeds:
- PostgreSQL Indexing: Create an HNSW index on the vector database table to optimize similarity searches:
CREATE INDEX ON meetings USING hnsw (embedding vector_cosine_ops);
- MongoDB Indexing: Create an Atlas Vector Search index on the
embeddingarray field, usingcosinesimilarity with 384 dimensions.
To protect external models (Hugging Face, Groq, Gemini) from API rate limits, the system uses rate-limiting middleware:
- IP-Based Throttling: Restricts clients to 60 ingestion requests per minute.
- Queue-Based Processing: Converts incoming ingestion tasks into sequential items in local memory, preventing server overloads.
We implemented three performance improvements in the ingestion pipeline:
- Optimization 1 (Vector Caching): Calculates embeddings only on updated or new transcript segments, saving redundant API calls.
- Optimization 2 (Client Compression): Compresses the base64-encoded audio payload using standard client gzip routines before transmission.
- Optimization 3 (Prefetching & Dynamic Bundling): Divides page views in Vite (
vite.config.ts) using lazy loading, allowing components to load on-demand.
Echo Cortex uses a high-contrast style guide optimized for long reading sessions:
:root {
--background-obsidian: #050505; /* Dark foundation background color */
--background-carbon: #131916; /* Mid-tone container background */
--text-crystal: #faf8f5; /* Crisp text highlight color */
--text-silver: #bfc6c4; /* Muted body text */
--color-neongreen: #00ff41; /* System health & active trace elements */
--color-cortex: #00e5ff; /* Core semantic highlights */
--color-amber: #f2a65a; /* Pending tasks and warnings */
--font-sans: 'Inter', system-ui; /* Primary typography font family */
--font-mono: 'JetBrains Mono'; /* Code, logs, and telemetry font family */
}Apply these variable tokens across your stylesheets to maintain interface consistency.
| Question | Solution |
|---|---|
| How does the system handle database outages? | The application automatically switches to local memory mode, saving new transcripts in a volatile array until connections are restored. |
| What audio formats are supported? | The pipeline accepts WebM, WAV, MP3, and OGG formats. We recommend WebM at 48kHz for optimal performance. |
| Why is the Whisper API returning placeholder transcripts? | This fallback is triggered if API key tokens are missing or if the external service experiences a cold start, keeping the ingestion workflow active. |
| Can I clear cached data locally? | Yes, you can clear the browser's storage by running localStorage.removeItem('echo_meetings') in the console. |
If you encounter runtime errors during ingestion, use this diagnostics checklist:
| Error Log | Probable Cause | Actionable Solution |
|---|---|---|
TypeError: fetch failed |
Express backend server is offline or port 2348 is blocked. |
Verify backend status via pm2 status or run npm run dev in the backend directory. |
Mongoose CastError |
Invalid MongoDB ObjectId query on local memory fallback assets. | Verify ID prefix. If using local caches, bypass Mongoose queries and retrieve data from inMemoryMeetings. |
Hugging Face Inference Warm-Up |
Whisper model is loading into memory on remote host. | Wait 30 seconds and retry ingestion. The pipeline will automatically use mock transcripts in the interim. |
JWT Expired |
Client Firebase auth token has expired. | Re-authenticate the client user session to generate a new ID token. |
We welcome contributions to Echo Cortex. Please read our design guidelines and follow this workflow:
- Fork the repository.
- Create a branch for your feature:
git checkout -b feature/your-feature. - Ensure your code matches the Obsidian Stark theme tokens and passes the endpoints test suite.
- Submit a pull request.
Echo Cortex projects follow standard open-source behaviors. Contributors are expected to maintain professional communication, review pull requests thoroughly, and respect the privacy policies established in the ingestion workflows.
We enforce semantic versioning and descriptive commit message patterns. Please prefix commit summaries with:
feat:for new UI panels, modules, or database controllers.fix:for bug resolutions (e.g. handling Mongoose ObjectId errors).docs:for README improvements, code descriptions, or deployment write-ups.style:for layout updates, CSS changes, or design system refinements.
This system is licensed under the MIT Open Source License. You are free to modify, deploy, and distribute it for private or organizational use.