SandboxVM — A Soft Sandbox for AmigaOS 4

SandboxVM lets you run AmigaOS 4 PowerPC binaries — programs, libraries, and drivers — inside a process-level sandbox, so a guest crash takes down the guest, not the host. The guest's entire world (heap, private IExec, library shims, comms rings, code image) lives in extended memory above the 2 GB barrier; the host sees the guest only through a controlled API surface.

Architectural deep-dive: see DESIGN.md.

What it is

A single OS4 host program (bin/sandboxvm) that:

• Allocates an ASOT_EXTMEM window above 2 GB for each guest.
• Maps that window into a slot-managed virtual address range and carves a slab+free-list heap out of it.
• Builds a private ExecIFace clone where the dangerous slots (memory, libraries, tasks, devices, IO dispatch) are overridden to track everything the guest opens, refuse foreign-handle closes, downgrade Forbid()/Permit() to a per-guest mutex, and route IO through a sandboxed dispatch path.
• Spawns the guest as a normal AOS4 Process with a tc_TrapCode trampoline that converts a wild store / illegal instruction / alignment fault inside the guest into a clean shutdown signal for the host.
• Reaps every guest resource (files, locks, ports, mutexes, IORequests, semaphores, library opens, signals, extmem regions, sub-tasks) via a per-guest ledger when the guest exits or aborts.

What it does

End-to-end on real X5000 hardware (production AOS4.1 FE kernel, 4 GB physical RAM, ExtMem enabled):

• Runs standard AOS4 ELF programs. Newlib- and clib4-linked C programs boot through the standard _start(args, arglen, sysbase) convention, run OpenLibrary("dos.library") → GetInterface("main") → main → exit, and clean up.
• Routes guest stdio. Per-guest T:sandboxvm-<name>.{out,err} capture, slurped to host stdout/stderr after the guest exits, with per-guest tags so multi-guest output stays attributable.
• Survives every kind of guest crash. Wild stores, NULL derefs, illegal instructions, privilege violations, stack overflows in the guest leave the host process untouched; subsequent guests run cleanly.
• Loads real production OS4 drivers. sandboxvm -r driver.device applies all PPC ELF relocations, scans the loaded image for an RTF_AUTOINIT Resident, parses the CLT_DataSize / CLT_InitFunc / CLT_Interfaces tag list, and invokes the driver's init function with the sandboxed IExec.
• Synthesises a virtio-scsi PCI device. When the driver walks PCI for vendor 0x1AF4/device 0x1004, it gets back a software-modeled PCIDevice * whose config space, BARs, legacy- virtio register state machine (STATUS / FEATURES / QUEUE_SEL / QUEUE_NUM / QUEUE_PFN / QUEUE_NOTIFY / ISR), and virtqueue processing all run inside the host process. The driver's DiscoverUnits round-trips 64 SCSI INQUIRYs through the synth vring during init and finds a fake DIRECT_ACCESS unit at target=0/lun=0 ("VMOS4 Virtual Disk 0001").
• Drives the loaded driver from a follow-on guest. sandboxvm -r driver.device -t test.elf runs test.elf as a second guest in the same Guest, so it can OpenLibrary the driver by name, OpenDevice it, allocate ASOT_PORT + ASOT_IOREQUEST, and drive the device via IExec->DoIO — all routed through the sandbox.

What it doesn't do

Concrete gaps and intentional non-goals — read these before assuming a use case fits:

• No MMU-level isolation. Guest and host share an address space. A *(int *)0xDEADBEEF = 42 in the guest still corrupts host memory; the sandbox catches the fault that follows (via tc_TrapCode), not the wild store itself. If you need true memory isolation, run a full VM (QEMU, real OS4 box, etc.).
• HD_SCSICMD via IExec->DoIO is not supported. The driver's _manager_BeginIO takes a libcall-cross-module edge into a kernel kmod that traps with a NULL-deref signature; resolving it needs a runtime PPC disassembler at the trap site. TD_MOTOR and any other inline-reply command go through cleanly via IExec->DoIO. See the in-tree TODO comment above priv_BeginIO for the full investigation context.
• No support for non-ExtMem hosts. Pegasos II is refused at startup. The whole guest world living above 2 GB is the central design choice — it isn't optional.
• One guest at a time. Multi-guest invocations run sequentially, not in parallel. The host process is single-threaded except for the per-guest worker task.
• DMA from extmem buffers is not real. priv_StartDMA / priv_GetDMAList / priv_EndDMA synthesise an identity mapping for vmem-owned (extmem) buffers — the kernel refuses to DMA-map extmem anyway. For real bus-master DMA, allocate the buffer with the SBV_AVT_HostDMA tag (see include/sbvm_tags.h); the override routes the alloc to the host's normal heap, and StartDMA / GetDMAList forward to the real IExec so the device gets real bus addresses. Fine for software-modeled synthetic devices either way (virtio-scsi takes the synth path); the new tag enables real PCI driver development.
• No multi-driver loading per Guest. -r driver.device loads exactly one resident driver into a Guest. There's no way today to bring up a SCSI driver and then a filesystem on top of it inside the same Guest.
• No live stdio streaming. Guest stdout/stderr is captured to T:sandboxvm-<name>.{out,err} and slurped to the host after the guest exits. Long-running guests don't get their output streamed back in real time (parked behind a workload that needs it).
• No interface wrapping for utility / graphics / etc. clib4's OpenLibrary("dos.library") goes through the dos shim, but utility.library, graphics.library, and any other library not in the per-Guest shim registry resolves to the host's interface verbatim. Calls into those bypass the sandbox. Add a shim if you need the boundary to extend there.
• Not a debugger / profiler / tracer. sandboxvm emits per-call debug prints to the kernel debug serial, but it has no breakpoint, step, or sampling support. Use IDebug or a real debugger for that.

Who should use it

SandboxVM is most useful if you:

• Are writing or maintaining an AmigaOS 4 PPC driver (.device / .library) and want to iterate on its init / discovery / I/O paths without cold-rebooting the host every time it crashes.
• Want a CI-style harness for OS4 binaries: spawn N guests in one host invocation, get tagged stdio per guest, exit codes surfaced to the shell, host process survives any failure.
• Are writing a test program that exercises sandboxed semantics: foreign-handle refusal, ledger-driven leak reaping, trap recovery after *(int *)0 = 1.
• Need a place to prototype synthetic PCI / virtio devices for driver development before real hardware is available.

It is not a replacement for QEMU or a full OS4 VM — guests run on a live OS4 install, sharing the kernel and most of the address space. The boundary is the API surface (the cloned IExec), not the MMU.

Requirements

• An OS4 system with ExtMem support: AmigaOne X5000 / X1000 / A1222 or QEMU AmigaOne with the ExtMem-capable kernel. Pegasos II is refused at startup (no ExtMem).
• Physical RAM large enough to back the guest's -m-size reservation (default 1 GiB) plus the normal AOS pool.
• For builds: Docker Desktop with the walkero/amigagccondocker:os4-gcc11 image (provides ppc-amigaos-gcc 11.5).
• The AmigaOS 4 SDK unpacked locally (default location: ~/sdk — override with SANDBOXVM_SDK_DIR).

The build runs on Windows (MSYS2 / Git Bash), macOS, or Linux — anywhere Docker Desktop runs. Compiled binaries are PPC ELF EXEC big-endian and run on the AOS4 target.

Installing the toolchain

# 1. Install Docker Desktop and start it.
# 2. Pull the OS4 cross-compile image:
docker pull walkero/amigagccondocker:os4-gcc11

# 3. Get the AOS4 SDK (from Hyperion's downloads page) and unpack
#    it locally. Default expected location: ~/sdk
#    Layout the build expects:
#      ~/sdk/include/include_h/{exec,dos,intuition,...}
#      ~/sdk/include/netinclude/...
#    Override the location via:
export SANDBOXVM_SDK_DIR=/path/to/your/sdk

# 4. Clone this repo wherever you keep projects.
git clone <this-repo> SandboxVM && cd SandboxVM

The docker-cc script in the repo wraps docker run with the right mount points and compiler invocation; you don't need to use Docker directly.

Building

make            # builds bin/sandboxvm + every guest test in test/
make host       # just the sandbox host (bin/sandboxvm)
make guest      # just the bundled freestanding hello-world (bin/hello)
make clean      # remove .o + bin/ artefacts
make help       # show all targets and env overrides

Produced binaries land in bin/:

• bin/sandboxvm — the host sandbox.
• bin/hello, bin/auxwin, bin/dostest, bin/inttest plus their -leak variants — freestanding C smoke tests that exercise the comms ring, dos.library shim, intuition.library shim, and the resource ledger on guest abort.
• bin/clib4hello, bin/clib4_argv, bin/clib4_file, bin/clib4_alloc, bin/clib4_atexit, bin/clib4_cwd, bin/clib4_crash, bin/clib4probe — -mcrt=clib4 programs that boot through the full AOS4 standard-toolchain crt0 chain.
• bin/crashy, bin/denytest — deliberate crash + library deny-list verifications.
• bin/virtioscsi-test — driver round-trip test against the synthetic virtio-scsi device.

clib4 programs need clib4.library available on the target; the simplest setup is Copy bin/clib4.library RAM: followed by assign LIBS: RAM: ADD from a shell.

Running

Copy bin/sandboxvm and any test guests onto the OS4 target (e.g. into RAM:), then from a Shell:

> sandboxvm RAM:hello                                # one guest
> sandboxvm RAM:hello RAM:dostest RAM:auxwin         # sequential
> sandboxvm RAM:clib4hello -- -v --opt 3             # CLI args via "--"
> sandboxvm -r RAM:nvme.device                       # driver-init only
> sandboxvm -r RAM:virtioscsi.device \
         -t RAM:virtioscsi-test                   # init + follow-on test
> sandboxvm -x utility.library RAM:hello             # deny a library

sandboxvm with no args (or unknown args) prints the full flag reference. Per-guest debug traces are written to the X5000 kernel debug serial via IExec->DebugPrintF; sandbox-level events ([sandboxvm] ...) go to a per-invocation log at RAM:sandboxvm-<task-ptr>.log.

Talking to the host from inside the sandbox

Guests call OpenLibrary("vm.host.library", 0) + GetInterface("main", 1, ...) to reach the synthesised IVMHost, then negotiate a pair of SPSC ring buffers for bulk data:

struct Library     *lb      = IExec->OpenLibrary("vm.host.library", 0);
struct VMHostIFace *IVMHost = (struct VMHostIFace *)
                              IExec->GetInterface(lb, "main", 1, NULL);
IVMHost->Hello(VMHOST_ABI_VERSION);
IVMHost->Log("guest is alive");

BYTE bit = IExec->AllocSignal(-1);
uint8  host_bit;
struct VMRing *g2h, *h2g;
IVMHost->RingNegotiate((uint8)bit, &host_bit, &g2h, &h2g);
vmring_push(g2h, VMR_VERB_PING, 0, "hello", 6);
IVMHost->Doorbell();
IExec->Wait(1u << bit);

Method-call control plane via IVMHost, batched bulk data via vmring_push / vmring_pop with a single Signal() doorbell. Both the rings and the IVMHost interface live in the guest's extmem heap. See DESIGN.md §8b for the protocol verbs, dead-guest detection, and PPC memory-ordering notes.

What is and is not protected

The sandbox lives at the API boundary, not the memory boundary. We do not own the MMU, so a wild store from guest code can still hit a host address.

What we catch:

• FreeVec() / CloseLibrary() / CloseDevice() of a foreign or already-freed handle (the resource ledger validates).
• Resource leaks on guest abort — every kernel object opened through the private IExec is reaped on exit.
• Forbid()/Permit() wedges (downgraded to a per-guest mutex).
• ColdReboot(), Supervisor(), SetIntVector() are refused.
• DSI / ISI / alignment / illegal-instruction / privilege traps in any guest task — converted to a clean shutdown signal via tc_TrapCode. Driver-spawned subtasks (e.g. virtio-scsi unit tasks) and IRQ-dispatch worker tasks (created by priv_AddIntServer) get the same coverage via priv_CreateTask's trampoline. Trap handler also fingerprints the kmod-libcall NULL+4 signature for a clear diagnostic when a guest hits the documented HD_SCSICMD-style limit.
• Real PCI bus-master DMA for buffers tagged SBV_AVT_HostDMA: routed to the kernel's real StartDMA instead of the synth identity map. Lets a real graphics / NIC / NVMe driver hand the device addressable bus pointers.
• Real PCI/MSI interrupts via priv_AddIntServer. The host ISR signals a per-registration worker task; the worker runs the guest's is_Code in normal task context with tc_TrapCode installed, so wild stores in IRQ handlers are sandboxed.

What we don't catch:

• *(uint32_t *)0xDEADBEEF = 42 from the guest. Wild stores still hit the host's address space; only a real MMU-boundary VM can fix that. The trap trampoline catches the consequence (the exception the guest takes when its own state gets corrupted), not the wild store itself.

See DESIGN.md §3 for the honest-limits writeup.

Future / planned

Roughly ordered by concreteness. None of these are funded work; they are extensions that fit the sandbox shape naturally.

• HD_SCSICMD via IExec->DoIO. The driver's _manager_BeginIO takes a libcall-cross-module edge into a kicklayout kmod that traps with a NULL-deref signature. Resolving it needs either a runtime PPC disassembler at the trap site (no GDB-stub channel on real X5000) or AOS4 PPC libcall function-descriptor docs we haven't extracted from the SDK. Once unblocked, the existing synth-vring path is ready to round-trip real SCSI commands through the driver. See the in-tree TODO above priv_BeginIO.
• Cooperative paging refinements. IVMHost->Pin(va) / Unpin(va) so guests can mark hot ranges; window eviction prefers cold-unpinned slots over LRU. Plus a free-list of unmapped backing offsets so unmapping a slot doesn't permanently lose its backing range, and contiguous N-slot grants for allocations larger than WINDOW_SLOT_SIZE. Lands when an actual paging workload demands it.
• Concurrent multi-guest. Today guests run sequentially. The Guest data structures are independent, so spawning N guests as parallel processes is mostly a main.c change plus a shared exit-code aggregator. Useful for stress testing the sandbox itself.
• Multi-driver per Guest. sandboxvm -r drv1 -r drv2 -t test — load multiple resident drivers into one Guest so a test program can mount a filesystem on top of a SCSI / SATA driver. Needs a driver-dependency story (-r order + symbol cross-resolution).
• More synthetic devices. The virtio-scsi pattern in src/shims/pci_synth.c extends naturally to other PCI device classes: virtio-net, virtio-block, NVMe, AC97. Each one needs a config-space + BAR + per-class state machine, ~500-1000 LOC.
• Real-time stdio relay. Pipe-based relay so a long-running guest's output streams back to the host as it's produced (instead of T:-file slurp at exit). Wants either a PIPE: handle or a host-side drain task watching the guest's output BPTR.
• Deeper graphics.library / utility.library overrides. The pass-through clones for both are in (src/shims/{graphics,utility}_shim.c) with ledger tracking and trailer guards on DropInterface, but individual methods are not yet sandboxed. Adding bitmap-residency checks on BltBitMap* (refuse blits onto host bitmaps from a guest-owned source, etc.) is the natural next layer when a real graphics driver starts running rendering loads.
• CLT_Vector68K legacy jump table. Process the driver's CLT_Vector68K tag and emit a jsr -N(libBase) style negative- offset thunk table so legacy 68k callers (or any code that reaches the driver via -30(io_Device) instead of an interface) works. Modern AOS4 callers go through IExec->DoIO and don't need it; this is for completeness.
• IDebug integration. Hook priv_iexec into IDebug so the host can dump live guest state (open libraries, ledger contents, vmem fragmentation) without recompiling. Useful as a runtime diagnostic surface.
• Driver-iteration CI harness. A wrapper script that: rebuilds a driver under test, uploads the binary to the X5000 via the amiga-fleet MCP, runs sandboxvm -r driver -t test, parses the trace, surfaces pass/fail. Closes the loop from source-edit to verified-run-on-real-hardware.

License

BSD 3-Clause. See LICENSE for the full text.

Repo layout

SandboxVM/
├── DESIGN.md            Architecture deep-dive
├── Makefile             Build, drives docker-cc
├── docker-cc            Wrapper: ppc-amigaos-gcc inside a Docker image
├── README.md            This file
├── include/             Public headers (vm.h, vmem.h, shim.h, ...)
├── src/
│   ├── main.c           Host entry + CLI
│   ├── vmem.c           Slab/free-list heap inside the extmem window
│   ├── window.c         Slot-managed view of the extmem mapping
│   ├── private_iexec.c  ExecIFace clone + trailer recovery
│   ├── shim.c           Per-guest shim registry
│   ├── elf_loader.c     ELF32 PPC loader (rels, _SDA_BASE_ resolve)
│   ├── resident_scan.c  RTF_AUTOINIT init + CLT_Interfaces processing
│   ├── guest_task.c     Lifecycle, ledger, trap trampoline, runner
│   ├── overrides/       Cloned IExec method implementations
│   ├── shims/           dos / intuition / expansion / pci_synth shims
│   └── vmhost*.c        IVMHost interface + ring comms + worker
└── test/                Guest test programs