Matrix

A small artificial society you can watch, question, and understand.

Explore the live demo · Operation and limits · Original project video

Matrix lets you explore how a fictional community responds to a shared proposal. Start a study, advance its social graph one round at a time, inspect what each resident heard, and ask an individual about their position. Static profile portraits keep the people visible; optional push-to-talk adds an AI voice to the conversation.

The current demo has two modes:

Mode	What runs	What you need
Preview	A deterministic simulation with preset fictional residents and scripted responses	The local backend and frontend; no API key
Live	OpenAI generates residents' responses and updated positions from their profiles, recent memories, and neighbors	A server-side OpenAI API key and available demo quota

Preview is explicitly labeled. Its dialogue is not presented as a live model response. Both modes are exploratory software demonstrations, not measured human behavior or validated forecasts.

Run locally

Use Python 3.11+ and Node.js 22.12+ (recommended; Node 20.19+ also works). The demo has a separate dependency file from the original research prototype.

From the repository root:

python3 -m venv .venv
.venv/bin/python -m pip install -r requirements-demo.txt
.venv/bin/uvicorn backend.demo_api:create_app --factory --host 127.0.0.1 --port 8000

In a second terminal:

cd frontend
npm ci
npm run dev

Open the local address printed by Vite. The frontend sends /api requests to the local backend. Preview works without an OpenAI key.

For Live mode, set OPENAI_API_KEY in the backend's environment before starting it. Never put this key in a VITE_ variable or in frontend code. The default text model is gpt-4.1-mini.

You can also copy .env.example to .env, add the key there, and pass --env-file .env to Uvicorn. If you have uv installed, ./start.sh installs the local dependencies when needed and starts both services, loading .env automatically.

The voice option uses gpt-4o-mini-transcribe for speech input and gpt-4o-mini-tts for generated speech. It requires a server-side key, microphone permission, and HTTPS or localhost. Text chat remains available without a microphone. The voice is AI-generated; this is turn-based push-to-talk, not a realtime video call.

How the demo works

React / Vite browser
        │  create · read · advance one round · chat · voice
        ▼
FastAPI demo service
        ├── SQLite: run state, ownership, operation leases, usage limits
        └── OpenAI: live text, transcription, generated speech

The hosted architecture uses Vercel for the frontend and a CPU-only Modal ASGI service with a persistent volume for SQLite. Modal hosts the API; OpenAI provides inference. No GPU allocation or model-weight download is needed for this demo.

Each advance request performs one simulation round. There is no long-running background simulation to leave running after the browser closes. The graph uses reproducible connections between fictional residents, and each round uses the previous state as its starting point. The interface exposes the graph, individual positions, and conversation history so you can inspect why a change occurred.

A run is limited to 12 residents and 5 rounds. The service also limits live runs, model calls, tokens, chat messages, and voice use. These are deployment limits, not a promise that model calls are free. Operator configuration and the session/recovery contract are documented in the demo guide.

Verify it

.venv/bin/python -m pip install pytest pytest-asyncio
.venv/bin/python -m pytest backend/tests -q
node --test frontend/tests/client-contract.test.mjs
npm --prefix frontend run build

The backend checks run ownership, persisted usage limits, state isolation, round recovery, provider errors, and bounded audio handling. Frontend contract checks cover PCM audio encoding, request credentials, expired sessions, and response handling. CI runs the backend suite and builds the frontend.

Original project and authorship

Originally built as a team project with aryan-cs/matrix; Bowen Zhu's fork preserves the original contributions and adds the current demo refresh.

• Original Devpost submission and video demo
• The original prototype explored Exa retrieval, Supermemory, Modal-hosted inference, and embodied avatar conversations.
• PLAN.md, project.md, and the older backend/avatar scripts describe that earlier implementation and research direction. Follow this README for the current demo.

The profile images are static previews from the original avatar catalog. Their original URLs are recorded in frontend/public/portraits/sources.json. They illustrate fictional characters; the demo does not start a HeyGen video session.

The old multi-GPU counts, Modal-credit estimates, and simulation runtime figures do not describe this OpenAI-backed demo. Current latency and cost depend on the selected mode, workload, API responses, and deployment environment.