Personal Knowledge Management & Learning System
"Tell me and I forget, teach me and I may remember, involve me and I learn." — Benjamin Franklin
A comprehensive system for ingesting, organizing, connecting, and actively learning from personal and professional knowledge sources—powered by LLMs and graph-based knowledge representation.
- Vision
- Core Philosophy
- System Architecture
- Ingestion Pipelines
- Organization Strategy
- Learning & Deliberate Practice
- Technical Stack
- Web Application
- Mobile Capture (PWA)
- Getting Started
- Implementation Status
- Documentation
- Open Research Questions
- Future Extensions
- Production Deployment
- Contributing
- Security
- License
- References
Transform passive information consumption into active knowledge acquisition through:
- Automated ingestion of diverse data sources
- Intelligent summarization and connection discovery
- Deliberate practice via AI-generated exercises and spaced repetition
| Challenge | Focus | Solution Approach |
|---|---|---|
| Extraction & Summarization | LLM-powered | Automated pipelines that distill raw sources into structured, interconnected notes |
| Learning & Retention | Human-centered | Active exercises, spaced repetition, and deliberate practice systems |
These challenges can be addressed independently, but solving extraction in service of learning maximizes value. Every piece of ingested content should feed into the learning loop.
┌─────────────────────────────────────────────────────────────────────────────┐
│ DATA SOURCES │
├─────────────┬─────────────┬─────────────┬─────────────┬─────────────────────┤
│ Papers │ Articles │ Books │ Code │ Ideas & Notes │
│ (Books.app)│ (Raindrop) │ (Physical) │ (Git repos) │ (Manual input) │
└──────┬──────┴──────┬──────┴──────┬──────┴──────┬──────┴──────────┬──────────┘
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ INGESTION LAYER │
├─────────────────────────────────────────────────────────────────────────────┤
│ • PDF Parser + Highlight Extractor + Handwriting OCR (Vision LLM) │
│ • Raindrop API Client │
│ • Book Photo OCR Pipeline (Mistral Vision API) │
│ • Git/GitHub API Integration │
│ • Manual/CLI Entry Tools │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ PROCESSING LAYER (LLM-Powered) │
├─────────────────────────────────────────────────────────────────────────────┤
│ • Summarization Engine │
│ • Key Concept Extraction │
│ • Tag & Topic Classification │
│ • Connection Discovery (semantic similarity) │
│ • Follow-up Task Generation │
│ • Exercise & Quiz Generation │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ KNOWLEDGE HUB (Obsidian) │
├─────────────────────────────────────────────────────────────────────────────┤
│ 📁 vault/ │
│ ├── 📁 sources/ # Raw ingested content organized by type │
│ │ ├── 📁 papers/ │
│ │ ├── 📁 articles/ │
│ │ ├── 📁 books/ │
│ │ ├── 📁 code/ │
│ │ └── 📁 ideas/ │
│ ├── 📁 topics/ # Topic-based index notes (auto-generated) │
│ ├── 📁 projects/ # Active learning projects │
│ ├── 📁 exercises/ # Generated practice problems │
│ ├── 📁 reviews/ # Spaced repetition queue │
│ └── 📁 meta/ # System config, templates, scripts │
└─────────────────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────────────────┐
│ KNOWLEDGE GRAPH (Neo4j) │
├─────────────────────────────────────────────────────────────────────────────┤
│ Nodes: Concepts, Sources, Topics, Authors, Tags │
│ Edges: RELATES_TO, CITES, CONTRADICTS, EXTENDS, PREREQUISITE_FOR │
│ Queries: "What do I know about X?", "What connects A to B?" │
└─────────────────────────────────────────────────────────────────────────────┘
│
┌───────────────┴───────────────┐
▼ ▼
┌──────────────────────────────────┐ ┌──────────────────────────────────────┐
│ LEARNING SYSTEM (FSRS) │ │ AI ASSISTANT (LLM Agent) │
├──────────────────────────────────┤ ├──────────────────────────────────────┤
│ • Spaced repetition scheduling │ │ • Natural language chat interface │
│ • Flashcard management │ │ • RAG over vault & knowledge graph │
│ • Mastery tracking │ │ • Streaming responses with citations│
│ • Practice session orchestration│ │ • Context-aware follow-up questions │
│ • Weak-spot identification │ │ • Configurable LLM model selection │
└──────────────────────────────────┘ │ • Conversation history & sessions │
│ • Source linking to original notes │
└──────────────────────────────────────┘
Source: MacOS Books app, Zotero, direct PDF uploads
The PDF pipeline uses a hybrid approach:
- Mistral OCR – Single API call that extracts full document text (markdown-formatted with tables & figures) AND detects handwritten notes/diagrams via image annotations
- PyMuPDF – Separate pass to extract PDF annotation objects (highlights, underlines, comments, sticky notes) from the PDF structure
┌─────────────────────────────────────────────────────────────────┐
│ PDF INPUT │
└─────────────────────────────┬───────────────────────────────────┘
│
┌──────────────────┴──────────────────┐
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ Mistral OCR │ │ PyMuPDF │
│ (single API call) │ │ (PDF structure parse) │
├─────────────────────────┤ └────────────┬────────────┘
│ • Full text (markdown) │ │
│ • Tables & figures │ ▼
│ • Handwritten notes │ ┌─────────────────────────┐
│ (via image analysis) │ │ Digital Annotations │
│ • Diagrams detected │ │ • Highlights │
└────────────┬────────────┘ │ • Underlines │
│ │ • Comments/sticky notes│
│ └────────────┬────────────┘
│ │
└──────────────────┬──────────────────┘
▼
┌───────────────────────────────┐
│ Unified Content Merge │
│ (associate annotations with │
│ their context in the paper) │
└───────────────┬───────────────┘
│
▼
┌───────────────────────────────┐
│ LLM Summarization │
└───────────────┬───────────────┘
│
▼
┌───────────────────────────────┐
│ Markdown Note │
└───────────────────────────────┘
Output: Structured markdown with summary, key findings, highlights, handwritten notes with context, and auto-generated follow-up questions.
Source: Raindrop.io API
Raindrop Collection → API fetch → Content extraction →
LLM Summarization → Markdown note with highlights preserved
Integration Points:
- Scheduled sync (daily/hourly)
- Preserve Raindrop collections as Obsidian folders or tags
- Extract user highlights as blockquotes
- Archive original content (avoid link rot)
Source: Photos of highlighted pages
Photo → Mistral Vision OCR → Highlight extraction →
Text cleanup → LLM processing → Structured book notes
Workflow:
- Photograph highlighted pages with consistent lighting
- Batch process through OCR pipeline
- AI identifies highlighted vs. non-highlighted text
- Aggregate into chapter-based or theme-based notes
- Store original images in separate media vault
Source: GitHub starred repos, personal projects
Git repo → Structure analysis → README parsing →
Key file identification → LLM code summarization →
Markdown note with architecture overview, key patterns, learnings
Captured Elements:
- Repository purpose and architecture
- Notable design patterns
- Dependencies and technology stack
- Personal notes on why it was saved
- Code snippets worth remembering
Source: CLI tool, mobile app, voice memos
Quick capture → Inbox folder → Daily processing →
Elaboration or linking to existing notes
Quick capture sends items to an inbox folder for daily processing, elaboration, and linking to existing notes.
sources/
├── papers/ # Academic papers, research
├── articles/ # Blog posts, news, essays
├── books/ # Book notes and highlights
├── code/ # Repository analyses
├── ideas/ # Fleeting notes, thoughts
└── work/ # Meetings, proposals, slack
Hierarchical topic tags (ml/transformers, systems/distributed) and meta tags (status/actionable, quality/foundational).
Leverage Obsidian's [[wikilinks]] extensively:
- Every note should link to related concepts
- Use block references for granular connections
- Auto-generate backlink summaries
📖 Full details: See LEARNING_THEORY.md for research foundations and citations.
This system is grounded in research on human memory and learning. Key insights:
| Research | Key Finding | System Implementation |
|---|---|---|
| Ericsson (2008) — Deliberate Practice | Expertise requires structured practice with feedback, not just experience | Adaptive difficulty + immediate LLM feedback |
| Bjork & Bjork (2011) — Desirable Difficulties | Spacing, interleaving, and generation enhance long-term retention | Spaced repetition + varied exercises |
| Dunlosky et al. (2013) — Learning Techniques | Practice testing and distributed practice are highest utility; highlighting/rereading are lowest | Retrieval-based exercises, avoid recognition tasks |
| Van Gog et al. (2011) — Cognitive Load | Worked examples before problems for novices | Adaptive: examples → testing as mastery increases |
| Chi et al. (1994) — Self-Explanation | Prompting self-explanation builds correct mental models | Self-explanation prompts in exercises |
- Learning ≠ Performance: Easy recall during study (retrieval strength) doesn't guarantee long-term retention (storage strength)
- Generation over Recognition: Producing answers from memory beats re-reading or highlighting
- Desirable Difficulties: Spacing, interleaving, testing, and variation slow immediate performance but enhance retention
- Adaptive Scaffolding: Novices get worked examples; intermediates get retrieval practice
┌─────────────────────────────┐
│ 1. INGEST NEW CONTENT │
│ (automated pipelines) │
└─────────────┬───────────────┘
│
▼
┌─────────────────────────────┐
│ 2. UNDERSTAND & CONNECT │
│ (summarization, linking) │
└─────────────┬───────────────┘
│
▼
┌─────────────────────────────┐
│ 3. ACTIVE PRACTICE │◄────────┐
│ (generation, not review) │ │
└─────────────┬───────────────┘ │
│ │
▼ │
┌─────────────────────────────┐ │
│ 4. SPACED RETRIEVAL │ │
│ (testing > restudying) │─────────┘
└─────────────────────────────┘
| Content Type | Exercise Types | Desirable Difficulty Applied |
|---|---|---|
| Conceptual | Explain-in-own-words, compare/contrast, teach-back | Generation effect (no notes allowed) |
| Technical | Implement from scratch, debug code, extend functionality | Generation + Variation |
| Procedural | Reconstruct steps from memory, adapt to new scenario | Retrieval practice + Interleaving |
| Analytical | Case study analysis, predict outcomes, critique approaches | Generation + Spacing |
- Generate Anki-compatible flashcards from key concepts
- Schedule review sessions based on forgetting curves (FSRS algorithm)
- Track confidence levels per concept
- Surface weak areas for targeted practice
|
Python 3.11+ |
FastAPI |
React 18 |
Vite |
TailwindCSS |
PostgreSQL |
Neo4j |
Redis |
Docker |
Obsidian |
LiteLLM |
Celery |
Mistral OCR |
Gemini |
Anthropic |
OpenAI |
Raindrop.io |
GitHub API |
| Component | Technology | Rationale |
|---|---|---|
| Frontend | React + Vite + TailwindCSS | Modern, fast, great DX |
| Backend | FastAPI + Python | Async, type-safe, OpenAPI docs |
| Knowledge Hub | Obsidian | Markdown-based, local-first, extensible |
| Graph Database | Neo4j | Native graph storage, Cypher queries |
| Relational DB | PostgreSQL | Learning records, user data |
| Cache | Redis | Session state, rate limiting |
| Task Queue | Celery | Async background job processing |
| LLM Backbone | LiteLLM (GitHub) | Unified interface to 100+ LLMs (OpenAI, Anthropic, Gemini, Mistral) |
| Vision/OCR | Mistral OCR (default for PDFs), Gemini 3 Flash | Document processing, handwriting recognition |
| Service | Purpose |
|---|---|
| Raindrop.io | Web bookmark sync |
| GitHub | Repository analysis |
| Mistral | Primary OCR for PDF/document processing |
| Google (Gemini) | Default text LLM for summarization, exercises |
The Second Brain web application provides a full-featured interface for knowledge management, spaced repetition learning, and analytics.
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Frontend │────▶│ Backend │────▶│ Data Layer │
│ (React/Vite) │ │ (FastAPI) │ │ Neo4j/PG/Redis │
└─────────────────┘ └─────────────────┘ └─────────────────┘
Frontend Pages: Dashboard, Practice Session, Exercises Catalogue, Card Catalogue, Review Queue, Knowledge Explorer, Knowledge Graph, Analytics, Follow-up Tasks, LLM Usage, Learning Assistant, Settings
┌──────────────────────────────────────────────────────────────────────────────┐
│ FRONTEND (React) │
├──────────────────────────────────────────────────────────────────────────────┤
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────────────┐ │
│ │ Practice │ │ Review │ │ Analytics │ │ Knowledge │ │
│ │ Session │ │ Queue │ │ Dashboard │ │ Explorer │ │
│ └─────────────┘ └─────────────┘ └─────────────┘ └─────────────────────┘ │
│ │ │ │ │ │
│ • Free recall • Spaced cards • Learning curves • Graph viz │
│ • Self-explain • Due items • Topic mastery • Note browser │
│ • Worked examples• Confidence • Time invested • Connection map │
│ • Interleaved Qs ratings • Weak spots • Search │
└──────────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────────┐
│ BACKEND (FastAPI) │
├──────────────────────────────────────────────────────────────────────────────┤
│ /api/practice/* /api/review/* /api/analytics/* │
│ ├── generate-exercise ├── due-items ├── learning-curve │
│ ├── submit-response ├── update-card ├── topic-mastery │
│ ├── get-feedback ├── schedule ├── session-history │
│ └── self-explain └── confidence └── weak-spots │
│ │
│ /api/knowledge/* /api/ingest/* /api/assistant/* │
│ ├── graph ├── pdf ├── chat │
│ ├── search ├── raindrop ├── generate-questions │
│ ├── connections ├── ocr └── explain-connection │
│ └── topics └── github │
│ │
│ /api/capture/* │
│ ├── text, url, photo, voice, pdf, book │
└──────────────────────────────────────────────────────────────────────────────┘
│
▼
┌──────────────────────────────────────────────────────────────────────────────┐
│ DATA LAYER │
├─────────────────────────┬─────────────────────────┬──────────────────────────┤
│ Neo4j │ PostgreSQL │ Redis │
│ Knowledge Graph │ Learning Records │ Session Cache │
├─────────────────────────┼─────────────────────────┼──────────────────────────┤
│ • Concepts & relations │ • Practice attempts │ • Active sessions │
│ • Source documents │ • Confidence ratings │ • Temp exercise state │
│ • Topic hierarchies │ • Spaced rep schedule │ • Rate limiting │
│ • Semantic embeddings │ • Time tracking │ │
└─────────────────────────┴─────────────────────────┴──────────────────────────┘
Backend APIs: /api/practice/*, /api/review/*, /api/analytics/*, /api/knowledge/*, /api/ingest/*, /api/assistant/*, /api/capture/*
The application features a modern, dark-themed interface optimized for focused learning and knowledge management.
The Dashboard serves as your home screen, answering "What should I do today?" at a glance. It displays a stats header showing your current streak, due review cards, and daily progress toward your learning goals. Quick action cards provide one-click access to practice sessions and review queues. The page also surfaces due cards for immediate review, identifies weak spots requiring attention, shows a streak calendar for motivation, and includes a quick capture input for rapidly saving ideas or URLs.
The Practice Session page enables deep learning through structured exercises grounded in cognitive science. You can select topics hierarchically with visual mastery indicators showing your current level, configure session duration (5-30 minutes), and choose whether to reuse existing exercises or generate new AI-powered ones. Exercise types include free recall (retrieve from memory), self-explanation (explain concepts in your own words), worked examples (study solutions before attempting), code debugging, and teach-back prompts—all with immediate LLM-powered feedback on your responses.
The Exercises Catalogue provides a comprehensive browsing interface for all available exercises in the system. You can search exercises with full-text search, filter by exercise type (recall, explain, apply, code) and difficulty level, group by topic, and jump directly into practice mode. Each exercise card shows its type, difficulty, associated topic, and when it was last practiced, helping you identify fresh material or areas needing review.
The Card Catalogue lets you browse and manage all spaced repetition flashcards in your knowledge base. Cards are organized by topic and state (new, learning, review, mastered), with filters for card type (definition, comparison, application, example, concept). You can search cards, view their front/back content, see scheduling information, and track mastery progress. This page complements the Review Queue by providing a library view of all your cards rather than just those currently due.
The Review Queue implements evidence-based spaced repetition using the FSRS (Free Spaced Repetition Scheduler) algorithm. Cards due for review are presented one at a time with active recall—you type your answer before seeing the correct response, which is more effective than simple recognition. The LLM evaluates your answers for semantic correctness, allowing for variations in wording. You then rate your confidence (Again/Hard/Good/Easy) to adjust scheduling. The page also supports AI-powered card generation from your notes.
The Knowledge Explorer provides a unified interface for browsing your entire knowledge base stored in Obsidian. Toggle between tree view (folder hierarchy) and list view (flat listing), use real-time search to find notes instantly, and access the command palette with ⌘K for quick navigation. Selected notes render inline with full markdown support including syntax highlighting for code blocks, LaTeX math, and wiki-link navigation. Deep linking support means you can share URLs to specific notes.
The Knowledge Graph offers an interactive D3.js force-directed visualization of your Neo4j knowledge graph. Different node types (Content, Concepts, Notes) are color-coded, with edges representing relationships like RELATES_TO, CITES, EXTENDS, and PREREQUISITE_FOR. Click nodes to view details, drag to rearrange the layout, and scroll to zoom in/out. A statistics sidebar shows content breakdown by type. This visualization helps discover unexpected connections between ideas and identify knowledge clusters.
The Analytics Dashboard provides comprehensive insights into your learning journey. The stats grid displays total time invested, current streak, and overall mastery percentage. Activity charts show practice sessions and review activity over configurable time periods (7/30/90 days). A topic mastery radar visualizes your proficiency across different knowledge areas. Progress breakdowns show completion by topic, while weak spots analysis identifies topics with declining retention and provides "Practice Now" buttons for targeted improvement. Calculated insights surface trends and recommendations.
The Follow-up Tasks page displays actionable items generated automatically during content processing. When you ingest a paper, article, or book, the LLM identifies potential follow-up actions: research topics to explore, concepts to practice, connections to make with other notes, and applications to try. Tasks are categorized by type (research, practice, connect, apply, review) and priority (high, medium, low), with estimated time requirements. You can filter, search, and mark tasks complete as you work through them, turning passive reading into active engagement.
The Ingest page provides a unified interface for capturing new content directly from the desktop web UI and monitoring the entire ingestion pipeline. The top panel offers tabbed capture for text notes, URLs, and file uploads (PDFs, images, audio) with expandable options for tagging, title, and learning material generation (cards/exercises). The bottom panel displays a live, auto-refreshing ingestion queue showing all content items across every status (Pending, Processing, Completed, Failed). Click any item to expand a detail panel with full processing stage progress, error messages, cost/token stats, and a direct link to the generated note in the Knowledge Explorer.
The LLM Usage page provides a dashboard for monitoring your AI API usage and costs. It displays budget status with visual progress bars, spending trends over time, and breakdowns by model (GPT-4, Claude, Gemini, Mistral) and pipeline (ingestion, processing, exercises, assistant). This transparency helps you understand where AI costs go and optimize your usage. You can set monthly budgets and receive alerts when approaching limits.
The Learning Assistant is an AI-powered chat interface for exploring your knowledge base conversationally. Ask natural language questions like "What do I know about attention mechanisms?" or "How does paper X relate to paper Y?" The assistant searches your vault and knowledge graph, synthesizes information, and provides source citations linking back to your notes. You can configure which LLM model to use, and responses stream in real-time with full markdown rendering. This turns your knowledge base into an interactive, queryable resource.
The Settings page lets you customize the application to your preferences. Appearance settings include compact mode for denser information display and animation toggles for reduced motion. Learning preferences let you set default session lengths and daily practice goals. Keyboard shortcuts are configurable, with defaults like ⌘K for command palette and ⌘1-6 for page navigation. You can also manage notification preferences, configure LLM model defaults, and export your data for backup or migration purposes.
A critical bottleneck in knowledge management is capture friction — the effort required to get information into the system. The companion Progressive Web App provides a mobile-optimized interface for on-the-go capture with offline support.
| Scenario | Capture Method | Processing |
|---|---|---|
| Physical book highlight | Photo of page | Vision OCR → highlight extraction → ingest |
| Fleeting idea | Voice memo or text | Transcription → LLM expansion → inbox |
| Interesting article | Share sheet / URL | Content fetch → summarize → save |
| Whiteboard / diagram | Photo | Vision LLM → describe → save with image |
| PDF document | File upload | Mistral OCR → full processing pipeline |
┌─────────────────────────────────────────────────────────────────┐
│ MOBILE DEVICE │
├─────────────────────────────────────────────────────────────────┤
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ 📷 Camera │ │ 🎤 Voice │ │ 📎 Share │ │
│ │ (book pages, │ │ (ideas, │ │ (URLs, │ │
│ │ whiteboards) │ │ memos) │ │ articles) │ │
│ └──────┬───────┘ └──────┬───────┘ └──────┬───────┘ │
│ └─────────────────┼─────────────────┘ │
│ ┌────────▼────────┐ │
│ │ PWA / Mobile │ │
│ │ Quick Capture │ │
│ └────────┬────────┘ │
└────────────────────────────┼─────────────────────────────────────┘
│ Upload (queue if offline)
▼
┌─────────────────────────────────────────────────────────────────┐
│ BACKEND │
├─────────────────────────────────────────────────────────────────┤
│ │
│ /api/capture/photo → Vision OCR → Text extraction │
│ /api/capture/voice → Whisper transcription → LLM expand │
│ /api/capture/url → Content fetch → Summarize │
│ /api/capture/text → Save to inbox → Tag suggestion │
│ /api/capture/pdf → Mistral OCR → Full pipeline │
│ /api/capture/book → Batch page OCR → Book notes │
│ │
│ ┌────────────────────┐ │
│ │ Inbox Processing │ │
│ │ (async via Celery) │ │
│ └─────────┬──────────┘ │
│ │ │
│ ┌──────────────┼──────────────┐ │
│ ▼ ▼ ▼ │
│ Neo4j Obsidian PostgreSQL │
│ (concepts) (notes) (metadata) │
└─────────────────────────────────────────────────────────────────┘
| Feature | Details |
|---|---|
| Installable | "Add to Home Screen" — launches like a native app |
| Offline capable | Service worker caches assets and queues captures in IndexedDB |
| Background sync | Automatically uploads queued captures when connection is restored |
| Share target | Receive shared URLs, text, and images from other apps (Android) |
| < 3 second capture | Minimal UI with large touch targets optimized for speed |
The PWA runs as a separate lightweight frontend (localhost:5174) and communicates with the same backend API. All captures are processed asynchronously via Celery and flow into the standard ingestion pipeline.
See 08_mobile_capture.md for full design details.
Left to right: Main capture screen with all capture types, Quick Note text capture, URL capture for saving links
- Python 3.11+
- Docker Desktop installed and running
- At least one LLM API key (Gemini, Mistral, OpenAI, or Anthropic)
# Clone the repository
git clone https://github.com/<your-username>/second-brain.git
cd second-brain
# Run the interactive setup script
python scripts/setup_project.pyThe setup script guides you through:
- Environment configuration — API keys, database credentials, data directory
- Vault setup — Obsidian folder structure, templates, meta notes
- Docker services — Start PostgreSQL, Neo4j, Redis, backend, frontend
- Database migrations — Initialize schema
python scripts/setup_project.py # Full interactive setup
python scripts/setup_project.py --non-interactive # Use defaults
python scripts/setup_project.py --env-only # Only configure .env
python scripts/setup_project.py --help-only # Show all available commands
python scripts/setup_project.py --help-env # Show env variable reference| Service | URL | Description |
|---|---|---|
| Frontend | http://localhost:3000 | Main web application |
| Knowledge Graph | http://localhost:3000/graph | Interactive graph visualization |
| Mobile Capture PWA | http://localhost:5174 | Mobile-optimized capture app |
| Backend API | http://localhost:8000 | REST API endpoints |
| API Documentation | http://localhost:8000/docs | Swagger/OpenAPI docs |
| Neo4j Browser | http://localhost:7474 | Graph database UI |
Backend: cd backend && pip install -r requirements.txt && uvicorn app.main:app --reload
Frontend: cd frontend && npm install && npm run dev
macOS
Prerequisites:
# Install Homebrew (if not installed)
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
# Install Python 3.11+
brew install python@3.11
# Install Docker Desktop
# Download from: https://www.docker.com/products/docker-desktop/
# Or via Homebrew:
brew install --cask docker
# Verify installations
python3 --version # Should be 3.11+
docker --version # Should show Docker version
docker compose versionNotes:
- Docker Desktop must be running before
docker composecommands - On Apple Silicon (M1/M2/M3), Docker automatically handles ARM64 architecture
- The
~tilde expands correctly on macOS for local development
Linux (Ubuntu/Debian)
Prerequisites:
# Update package list
sudo apt update
# Install Python 3.11+
sudo apt install python3.11 python3.11-venv python3-pip
# Install Docker (official method)
# Remove old versions
sudo apt remove docker docker-engine docker.io containerd runc
# Install prerequisites
sudo apt install ca-certificates curl gnupg lsb-release
# Add Docker's official GPG key
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
# Set up repository
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
# Install Docker
sudo apt update
sudo apt install docker-ce docker-ce-cli containerd.io docker-compose-plugin
# Add your user to the docker group (to run without sudo)
sudo usermod -aG docker $USER
newgrp docker
# Verify installations
python3 --version
docker --version
docker compose versionNotes:
- Log out and back in for docker group changes to take effect
- For systemd services, use absolute paths (tilde
~won't expand) - If running in WSL2, see Windows section for additional notes
Windows (with WSL2)
Prerequisites:
-
Install WSL2:
# Run in PowerShell as Administrator wsl --install # Restart your computer
-
Install Docker Desktop:
- Download from: https://www.docker.com/products/docker-desktop/
- During installation, enable "Use WSL 2 based engine"
- After installation, open Docker Desktop Settings → Resources → WSL Integration
- Enable integration with your WSL distribution
-
In WSL2 terminal (Ubuntu):
# Install Python sudo apt update sudo apt install python3.11 python3.11-venv python3-pip # Verify Docker (provided by Docker Desktop) docker --version docker compose version
Notes:
- Run all commands from within WSL2, not PowerShell
- Store your project in the WSL filesystem (
/home/user/) not/mnt/c/for better performance - Use absolute paths in
.envfile (e.g.,/home/user/datanot~/data) - Docker Desktop manages the Docker daemon; you don't need to start it manually
After installation, verify everything is working:
# Check Python version (should be 3.11+)
python3 --version
# Check Docker is running
docker info
# Check Docker Compose
docker compose version
# Test Docker can run containers
docker run hello-world
# Verify GPU support (optional, for local LLM inference)
docker run --rm --gpus all nvidia/cuda:11.8.0-base-ubuntu22.04 nvidia-smiDocker daemon not running
macOS/Windows: Start Docker Desktop application.
Linux:
sudo systemctl start docker
sudo systemctl enable docker # Start on bootPermission denied when running docker
Linux:
sudo usermod -aG docker $USER
# Log out and back in, or run:
newgrp dockerPort already in use
Check what's using the port:
# macOS/Linux
lsof -i :8000 # Backend
lsof -i :3000 # Frontend
lsof -i :5432 # PostgreSQL
# Stop the process or use different ports in docker-compose.ymlDatabase connection refused
- Check if containers are running:
docker compose ps - Check container logs:
docker compose logs postgres - Verify
.envfile has correct credentials - Wait for healthcheck to pass (can take 30 seconds)
Neo4j won't start / Out of memory
Neo4j requires significant memory. Ensure Docker Desktop has at least 4GB RAM allocated:
- Docker Desktop: Settings → Resources → Memory → Set to 4GB+
- Linux: Check available memory with
free -h
| Command | Purpose |
|---|---|
python scripts/pipelines/run_pipeline.py article <URL> |
Import web article |
python scripts/pipelines/run_pipeline.py pdf <file> |
Process PDF document |
python scripts/pipelines/run_pipeline.py book <file> |
OCR book photos |
python scripts/run_processing.py process-pending |
Process all pending content |
python scripts/run_all_tests.py |
Run all tests |
docker compose logs -f backend |
View backend logs |
docker compose down -v |
Stop and remove all data |
📁 Full Details: See
implementation_plan/OVERVIEW.mdfor the complete implementation roadmap with task checklists.
| Phase | Focus | Status |
|---|---|---|
| 1 | Foundation & Infrastructure | ✅ Complete |
| 2 | Ingestion Pipelines | ✅ Complete |
| 3 | LLM Processing | ✅ Complete |
| 4 | Knowledge Graph (Neo4j) | ✅ Complete |
| 5 | Backend API | ✅ Complete |
| 6 | Frontend Application | ✅ Complete |
| 7 | Learning System (Exercises + FSRS) | ✅ Complete |
| 8 | Analytics Dashboard | ✅ Complete |
| 9 | Mobile Capture (PWA) | ✅ Complete |
| 10 | Assistant Tool Calling | ⬜ Not Started |
| 11 | MCP Integration | ⬜ Not Started |
| 12 | Polish & Production | 🟡 In Progress |
Detailed technical specifications for each system component:
| Document | Description |
|---|---|
| 00_system_overview.md | High-level architecture and component interactions |
| 01_ingestion_layer.md | Content ingestion pipelines and formats |
| 02_llm_processing_layer.md | LLM integration, prompts, and processing stages |
| 03_knowledge_hub_obsidian.md | Obsidian vault structure and templates |
| 04_knowledge_graph_neo4j.md | Neo4j schema, queries, and graph operations |
| 05_learning_system.md | Exercises, FSRS algorithm, mastery tracking |
| 06_backend_api.md | FastAPI endpoints and data models |
| 07_frontend_application.md | React components and state management |
| 08_mobile_capture.md | PWA mobile capture workflow |
| 09_assistant_tool_calling.md | LLM agent with tool calling |
| 10_observability.md | Logging, metrics, and monitoring |
Step-by-step implementation guides with task checklists:
| Document | Description |
|---|---|
| OVERVIEW.md | Master roadmap with all phases |
| 00_foundation_implementation.md | Infrastructure setup |
| 01_ingestion_layer_implementation.md | Content ingestion |
| 02_llm_processing_implementation.md | LLM processing stages |
| 03_knowledge_hub_obsidian_implementation.md | Obsidian integration |
| 04_knowledge_graph_neo4j_implementation.md | Neo4j setup and queries |
| 05_learning_system_implementation.md | Learning system |
| 06_backend_api_implementation.md | API development |
| 07_frontend_application_implementation.md | Frontend development |
| 08_mobile_capture_implementation.md | Mobile PWA |
| 09_assistant_tool_calling_implementation.md | Assistant tool calling |
| tech_debt.md | Technical debt tracking |
| Document | Description |
|---|---|
| LEARNING_THEORY.md | Learning science research foundations |
| TESTING.md | Testing guide and best practices |
-
Human vs. Machine Connection-Making: To what extent should we outsource relationship discovery to AI vs. keeping it as a human cognitive exercise?
-
Information Overload: How do we prevent the knowledge base from becoming overwhelming? What pruning and archival strategies work best?
-
Exercise Quality: Can current LLMs generate exercises that genuinely challenge and teach, or do they tend toward superficial quizzes?
Enable the Learning Assistant to take actions through natural language requests, turning it from a Q&A interface into an interactive agent:
User: "Generate an exercise about attention mechanisms"
→ Assistant calls generate_exercise tool
→ Returns interactive exercise card inline in chat
Planned Tools:
| Tool | Description |
|---|---|
generate_exercise |
Generate adaptive exercise for a topic based on current mastery |
create_flashcard |
Create a spaced repetition card from conversation context |
search_knowledge |
Search the knowledge graph with natural language |
get_mastery |
Retrieve mastery state and learning history for a topic |
get_weak_spots |
Identify topics with declining retention needing review |
The design uses an LLM tool-calling loop: the model decides when to invoke tools, results are fed back for a synthesized response. See 09_assistant_tool_calling.md for the full design.
Expose the knowledge base as MCP servers so any MCP-compatible LLM client (Claude Desktop, Cursor, etc.) can directly query your Second Brain:
User (in Claude Desktop): "What do I know about distributed consensus?"
→ LLM queries Second Brain MCP server
→ Server searches Obsidian vault + Neo4j graph
→ Returns relevant notes with citations
Planned MCP Servers:
| Server | Capabilities |
|---|---|
| Vault server | Read/search/write Obsidian notes, list by topic |
| Knowledge graph server | Cypher queries, concept lookup, relationship traversal |
| Learning server | Exercise generation, spaced rep scheduling, mastery queries |
For production deployments, see the comprehensive guides in docs/deployment/:
| Document | Description |
|---|---|
| production.md | Full production deployment guide |
| security.md | Security hardening and best practices |
Key Production Steps:
- Configure environment — Set production values in
.env(disable debug mode, set real secrets) - SSL/TLS — Use Let's Encrypt with Certbot for HTTPS certificates
- Reverse proxy — Configure Nginx for rate limiting, security headers, and proxying
- CORS — Restrict
CORS_ORIGINSto your production domains - Database security — Strong passwords, network isolation, regular backups
- Container security — Run as non-root, set resource limits, use read-only filesystems where possible
Quick Production Checklist:
# Required environment changes for production
DEBUG=false
SECRET_KEY=<generate-secure-random-key>
CORS_ORIGINS=https://yourdomain.com
POSTGRES_PASSWORD=<strong-password>
NEO4J_PASSWORD=<strong-password>See production.md for complete instructions including Docker configuration, Nginx setup, backup procedures, and monitoring.
We welcome contributions! Please see our Contributing Guide for details on:
- Development environment setup
- Code style guidelines (Python and JavaScript/React)
- Commit message conventions
- Pull request process
- Testing requirements
Quick Start for Contributors:
# Fork and clone the repository
git clone https://github.com/<your-username>/second-brain.git
cd second-brain
# Run the setup script
python scripts/setup_project.py
# Create a feature branch
git checkout -b feature/your-feature-name
# Make changes, then submit a PRFor security-related concerns, please review:
- Security Hardening Guide — Production security best practices
- Vulnerability Reporting — If you discover a security vulnerability, please report it responsibly by emailing the maintainers directly rather than opening a public issue
Security Features:
- Configurable CORS origins (not wildcard in production)
- Environment-based secrets management
- Database credential isolation
- Rate limiting support
- Security headers via reverse proxy
This project is licensed under the MIT License — see the LICENSE file for details.
The MIT License is a permissive license that allows:
- ✅ Commercial use
- ✅ Modification
- ✅ Distribution
- ✅ Private use
With the only requirement being to include the license and copyright notice in copies.
📖 See LEARNING_THEORY.md for detailed research summaries.
Key sources: Ericsson (2008) on Deliberate Practice, Bjork & Bjork (2011) on Desirable Difficulties, Dunlosky et al. (2013) on Effective Learning Techniques.
- How to Take Smart Notes – Sönke Ahrens (Zettelkasten method)
This is a living document. As the system evolves, so will this design.