README.md

Documentation

Organized by project phase. Shared material (source-paper studies) applies to every phase.

Shared

Document	What it covers
Kafka Paper Study	8-question walkthrough of the Kafka paper: pull vs push, page cache, ISR, zero-copy
Kafka Section 5	Experimental results from the paper

Decisions

Document	What it covers
Decision Index	Architecture Decision Records — key design choices with commit links
ADR-001	RWMutex for partition reads
ADR-002	Synchronous architecture (no writer goroutine)
ADR-003	Sparse index with interval=1
ADR-004	fdatasync over fsync
ADR-005	28-byte fixed-size message header
ADR-006	Length-prefixed protobuf wire protocol
ADR-007	FNV-1a hash for key-based partition routing
ADR-008	Release closed segment file handles after recovery
ADR-009	Fuzz testing with committed seed corpus
ADR-010	Immutable closed segments (chmod 0444)

Phase 1: Single-node Log Engine

Document	What it covers
Implementation Plan	Build order, dependency graph between components
Server Design Decisions	6-question decision tree for the TCP server: connection model, wire protocol, request lifecycle, partition sync, batching, broker API
Control Flow Diagrams	Mermaid diagrams: full-stack produce/consume, server lifecycle, partition-level flows
Concurrency Model	Four-layer locking hierarchy, parallelism model, shutdown coordination, known limitations
Log Design Doc	Storage layout, message format, index strategy, concurrency model, failure modes (lives with the package it documents)

Phase 2: Exactly-once via 2PC

Document	What it covers
Design Notes	Consumer groups, replication, 2PC, file descriptor management
Exactly-Once Penalty	Performance cost of 2PC vs at-least-once

Repo Root

Document	What it covers
README	Project overview, design decisions, phases, running
Progress Tracker	Phase timeline, component status, test coverage, key learnings
Testing Guide	All test modes: unit, race, fuzzing, benchmarks, container integration test, validator