BangOS 💥

BangOS is an educational, lightweight 64-bit bare-metal operating system microkernel designed to run standard Linux binaries (compiled statically with the musl standard C library) directly on x86_64 UEFI hardware without requiring the Linux kernel.

The repository serves as a comprehensive, clean foundation for learning low-level OS development, 64-bit UEFI bootloading, x86_64 CPU initialization (GDT, TSS, IDT), 4-level paging with Demand Paging, FPU/SSE SIMD extensions, preemptive multitasking, and the Linux System Call ABI.

---

🌟 Core Subsystems & Key Features

• 64-bit UEFI Bootloader (BOOTX64.EFI): Loads directly via OVMF UEFI firmware, sets up graphics/serial console, reads initrd.tar from FAT32 partitions, queries memory maps, and executes ExitBootServices().
• In-Memory TarFS Ramdisk: Parses standard USTAR archives containing standalone 64-bit ELF binaries directly in physical RAM.
• x86_64 Core Microkernel:
  • GDT & TSS: Segment descriptors configured for Kernel space (0x08, 0x10) and User space (0x1B, 0x23) with dynamic RSP0 stack switching.
  • IDT & Exception Handling: 256 interrupt gates with dedicated Interrupt Stack Table (IST1) for fault safety (#DF, #GP).
  • 4-Level Paging & VMM: PML4, PDPT, PD, PT hierarchy with 4 GB identity mapping, physical frame bitmap allocator, and per-process Virtual Memory Areas (VMAs).
  • Demand Paging: Zero-fill on-demand page allocation handling Page Fault exceptions (#PF, Vector 14) for dynamic mmap().
  • FPU & SSE SIMD Extensions: %cr0 / %cr4 configured for 64-bit hardware floating-point execution (sqrt, double precision math, fxsave64/fxrstor64 context switching).
• Preemptive Multitasking & Multithreading:
  • 8254 PIT Timer Driver: Generates 100 Hz timer interrupts (10 ms time quantum) for Round-Robin process scheduling.
  • Process Lifecycle: SYS_FORK, SYS_CLONE (with CLONE_VM), SYS_EXECVE, SYS_WAIT4, and SYS_EXIT.
  • Fast Userspace Locking: SYS_FUTEX supporting FUTEX_WAIT and FUTEX_WAKE.
• Block Storage & ext2 Filesystem Engine:
  • ATA / IDE PIO Driver: Hardware controller probing, 16-bit PIO sector read/write with LBA28 and LBA48 support.
  • Virtual File System (VFS): Hierarchical VFS supporting multiple mountpoints (/ for TarFS ramdisk, /mnt/ext2 for persistent storage), path resolution, and file descriptors.
  • ext2 Filesystem Engine: Superblock parsing (0xEF53), block group descriptor tables, inode tables, direct/indirect block resolution, directory lookup, and block write allocation.
• VirtIO Network Driver & In-Kernel TCP/IP Stack:
  • PCI Bus Enumeration: Hardware configuration space scanner (0xCF8/0xCFC) and BAR decoding.
  • VirtIO-Net Driver: OASIS VirtIO specification driver with split virtqueues (16 RX descriptors + TX descriptor rings).
  • Full In-Kernel TCP/IP Stack: Layer 2 Ethernet II framing, Layer 2.5 ARP broadcast resolution and dynamic cache table, Layer 3 IPv4 routing and one's complement Internet checksums, ICMP Echo pinging with TSC microsecond latency timing, Layer 4 UDP multiplexing, and RFC 1035 UDP DNS resolution (10.0.2.3:53).
  • TCP Connection Engine: Finite state machine (CLOSED -> SYN_SENT -> ESTABLISHED -> FIN_WAIT -> CLOSED), 3-way handshake (SYN/SYN+ACK/ACK), sequence arithmetic, sliding window circular ring buffer, and stream segmentation.
  • POSIX Sockets ABI: System calls for SYS_SOCKET, SYS_CONNECT, SYS_SENDTO, SYS_RECVFROM, SYS_SHUTDOWN, SYS_BIND, SYS_LISTEN, SYS_GETSOCKNAME, SYS_GETPEERNAME, SYS_SETSOCKOPT, SYS_GETSOCKOPT, and SYS_POLL.
• Linux x86_64 Syscall ABI: Hardware syscall / sysret MSR interface (STAR, LSTAR, SFMASK, EFER.SCE) implementing over 35 POSIX syscalls (open, close, read, write, lseek, stat, fstat, getdents64, mmap, mprotect, munmap, brk, poll, ioctl, nanosleep, sysinfo, uname, arch_prctl, socket, connect, sendto, recvfrom, etc.).
• Two-Tier Test Verification:
  • Ring 0 ktest: In-kernel unit tests verifying memory allocators, page tables, TarFS, scheduler, ATA PIO, ext2, VirtIO-Net, checksums, ARP, and TCP.
  • Ring 3 POSIX Specs: Standalone userland test suites validating syscall safety, demand paging, process lifecycles, ext2 file operations, and POSIX stream/datagram sockets.
• Standalone Multi-ELF Userland:
  • /bin/init (PID 1): Interactive process supervisor and launcher.
  • /bin/calc: Standalone geometric calculator.
  • /bin/sysinfo: Standalone hardware and operating system report.
  • /bin/bench: Multi-phase CPU, FPU/SSE, dynamic memory, and demand paging benchmark suite.
  • /bin/tasks: Preemptive multitasking and concurrent computation workers demo.
  • /bin/threads: Multithreading, mutexes, counting semaphores, and futex synchronization suite.
  • /bin/disktool: Interactive storage explorer, superblock inspector, and persistence tester.
  • /bin/netfetch: Interactive network diagnostics and HTTP/1.1 web payload client.

---

📚 Technical Documentation Syllabus

The interactive documentation website is published online at https://vikman90.github.io/bangos/.

Comprehensive documentation explaining the theoretical concepts, hardware specifications, and code implementations is available in the docs/ directory:

Chapter	Title	Key Topics Covered
00 - Architecture Overview	System Architecture & Design Philosophy	Microkernel design, boot flow, Ring 0 vs Ring 3, memory map, subsystem interactions.
01 - UEFI Bootloading	64-bit UEFI Bootloading & Firmware Handoff	GNU-EFI, FAT32 ESP traversal, initrd.tar loading, memory map acquisition, ExitBootServices.
02 - GDT, TSS & IDT	Segmentation, GDT, TSS & Exception Handling	Long Mode segmentation, segment selectors, TSS rsp0/ist1, 256-gate IDT, 8259 PIC remapping.
03 - Paging & Memory Management	4-Level Paging, Allocators & Demand Paging	PML4/PDPT/PD/PT translation, 4GB identity mapping, frame bitmap allocator, VMAs, Demand Paging #PF.
04 - System Call ABI	System Call Subsystem (Linux x86_64 ABI)	Hardware MSRs (STAR, LSTAR, SFMASK), syscall_entry.s trampoline, exhaustive syscall reference table.
05 - Process Management & Scheduling	Process Management, Preemptive Sched & Futex	PCB (process_t), fork(), clone(), musl user stack, 100 Hz PIT timer, Round-Robin scheduler, futex.
06 - In-Memory TarFS & ELF64 Loader	In-Memory TarFS Ramdisk & ELF64 Loader	USTAR format, octal parsing, Elf64_Ehdr, Elf64_Phdr, PT_LOAD mapping, BSS zeroing.
07 - FPU & SSE Extensions	FPU & SIMD/SSE Floating-Point Support	CR0/CR4 control registers, fninit, fxsave64/fxrstor64 context switching, 16-byte stack alignment.
08 - Hardware Device Drivers	Hardware Device Drivers & Low-Level I/O	16550 UART serial COM1 driver, 8254 PIT timer, PS/2 keyboard controller, QEMU debug/poweroff ports.
09 - Userland Environment	Userland Runtime, Applications & Concurrency	Static musl-gcc, /bin/init supervisor, standalone ELFs, tui.h ANSI library, synch.h concurrency.
10 - Specification Testing & QEMU	Specification-Driven Testing & QEMU Harness	Ring 0 ktest unit tests, Ring 3 POSIX specification suites, headless QEMU TCP serial test runner.
11 - Extending BangOS	Extending BangOS Developer Guide	Step-by-step developer tutorial for adding new syscalls, drivers, userland tools, and test suites.
12 - ATA / IDE Storage Driver	ATA / IDE Storage Driver Architecture	Port I/O register maps, PIO mode, LBA28/48, sector read/write protocols, block device abstraction.
13 - VFS & ext2 Filesystem	Virtual File System & ext2 Filesystem Engine	Superblock, block groups, inodes, direct/indirect mapping, directory records, and POSIX syscall mappings.
14 - VirtIO Network Driver	VirtIO Network Driver & PCI Enumeration	PCI bus scanner, legacy VirtIO specification, split virtqueues, RX/TX descriptor rings, packet polling.
15 - TCP/IP Network Stack	In-Kernel TCP/IP Network Stack	Ethernet II, ARP dynamic cache, IPv4 routing, checksums, ICMP pinging, UDP, DNS, TCP 3-way handshake.
16 - Sockets & HTTP Client	Socket API, HTTP/1.1 Client & TLS/HTTPS Guide	POSIX socket syscalls, VFS wrapping, /bin/netfetch HTTP client, and educational TLS/HTTPS layering.
17 - Build System & Tooling	Build System, Tooling & Developer Workflows	Makefile targets, QEMU interactive mode, GDB debugging, Docker containers, and MkDocs.

---

🖥️ Multi-ELF Userland Environment

======================================================================
        BangOS (x86_64) - Bare Metal Kernel v0.3.0        
     PID: 1 (init) | RAM: 128 MB Total (126 MB Free) | Uptime: 0 s
======================================================================

Available Standalone Applications (Multi-ELF Ramdisk):

  [1] Geometric Calculator           (execve /bin/calc)
  [2] System Information & Uname     (execve /bin/sysinfo)
  [3] CPU FPU/SSE & Timer Benchmark  (execve /bin/bench)
  [4] Preemptive Multitasking Tasks  (execve /bin/tasks)
  [5] Multithreading & Mutex Sync    (execve /bin/threads)
  [6] Disk Explorer & Storage Mgr    (execve /bin/disktool)
  [7] Run Specification Test Suites  (execve /bin/test_*)
  [8] Network Fetch & HTTP Client    (execve /bin/netfetch)
  [9] Shutdown / Halt System         (exit)

Select an option [1-9]: 

---

🛠️ Requirements & Dependencies

On Debian/Ubuntu-based Linux systems:

sudo apt-get update
sudo apt-get install -y \
    build-essential \
    gnu-efi \
    nasm \
    qemu-system-x86 \
    ovmf \
    musl \
    musl-tools \
    e2fsprogs \
    tar \
    python3

---

🚀 Building and Running

1. Build the OS & Bootloader

Compile userland binaries, assemble initrd.tar, compile the EFI kernel binary, and generate the FAT32 boot filesystem at build/esp/:

make esp

2. Run under QEMU

Launch BangOS in QEMU using OVMF UEFI firmware:

make run-qemu

3. Automated Test Suite

Execute the headless integration test harness with TCP socket serial communication:

make test

4. Docker / macOS Cross-Compilation

For building and testing in isolated containers or on macOS:

# Build OS artifacts inside Docker
make docker-build

# Run automated integration tests inside Docker
make docker-test

# Open interactive bash in the build container
make docker-shell

5. Package Distribution Archives

Package the ESP partition and ext2 storage disk into release archives:

make dist

---

📁 Repository Structure

BangOS/
├── Makefile                     # Root build system
├── LICENSE                      # MIT License
├── README.md                    # Project overview and index
├── AGENTS.md                    # AI agent guidelines & engineering standards
├── boot/                        # 64-bit UEFI Bootloader
│   └── main.c                   # Bootloader entrypoint & firmware handoff
├── docs/                        # Subsystem technical documentation
│   ├── 00-architecture.md
│   ├── 01-uefi-bootloading.md
│   ├── 02-gdt-idt-tss.md
│   ├── 03-paging-and-memory.md
│   ├── 04-syscall-abi.md
│   ├── 05-process-multitasking-sched.md
│   ├── 06-elf-loader-tarfs.md
│   ├── 07-fpu-sse.md
│   ├── 08-hardware-drivers.md
│   ├── 09-userland-environment.md
│   ├── 10-testing-and-qemu.md
│   ├── 11-extending-bangos.md
│   ├── 12-ata-storage-driver.md
│   ├── 13-vfs-and-ext2.md
│   ├── 14-virtio-network-driver.md
│   ├── 15-tcpip-network-stack.md
│   ├── 16-socket-api-and-http.md
│   └── 17-build-system-and-tooling.md
├── include/                     # Kernel header definitions
│   ├── kernel.h                 # Boot structures and MSR macros
│   └── uefi.h
├── kernel/                      # 64-bit OS Microkernel
│   ├── main.c                   # Kernel main orchestrator & FPU init
│   ├── arch/x86_64/             # GDT, TSS, IDT, and Syscall assembly
│   ├── drivers/                 # 16550 UART, 8254 PIT, 8259 PIC, PS/2, QEMU
│   ├── fs/                      # In-Memory TarFS ramdisk driver
│   ├── lib/                     # Freestanding kstring library
│   ├── loader/                  # ELF64 binary loader
│   ├── mm/                      # PMM frame allocator, Paging & VMM
│   ├── process/                 # Process table, fork, clone, scheduler, futex
│   ├── syscall/                 # Linux x86_64 syscall dispatcher
│   └── tests/                   # Ring 0 in-kernel unit test suite (ktest)
├── scripts/
│   ├── run_qemu_test.sh         # Test execution shell wrapper
│   └── test_runner.py           # Automated TCP socket QEMU test harness
└── userland/                    # Standalone C Userland & initrd packaging
    ├── Makefile                 # Userland compilation & USTAR tar rules
    ├── include/                 # Userland headers (tui.h, synch.h)
    └── src/                     # Standalone applications & POSIX test suites

---

🗺️ Roadmap & Subsystem Status

• 64-bit UEFI Bootloader with memory map parsing & ExitBootServices
• x86_64 GDT, TSS (rsp0, ist1), IDT (256 gates), FPU/SSE SIMD support
• 4-Level Paging (CR3) with 4GB Identity Mapping & 4KB Bitmap Frame Allocator
• Virtual Memory Area (VMA) Manager & On-Demand Paging on #PF (Vector 14)
• In-Memory TarFS Ramdisk Driver (initrd.tar) & ELF64 Program Header Loader
• Multi-Process Lifecycle (SYS_FORK, SYS_EXECVE, SYS_WAIT4, SYS_EXIT)
• Preemptive Multitasking & Context Switching (8254 PIT 100 Hz / 10 ms time slice)
• Multithreading (CLONE_VM), Thread Local Storage (FS_BASE), and SYS_FUTEX
• Standalone Multi-ELF Userland Suite & POSIX Specification Test Harness
• Two-Tier Automated Verification Architecture (Ring 0 ktest + Ring 3 POSIX)
• ATA / IDE Storage Drive Driver, VFS Multi-Mount & ext2 Filesystem Engine
• VirtIO Network Driver & Lightweight TCP/IP Stack

---

📄 License

Distributed under the MIT License. See LICENSE for details.