ArgosFS

ArgosFS is a self-driving, erasure-coded filesystem with a Rust core and FUSE frontend. It now has three storage backends:

• HostFsBackend for development and compatibility, storing shards as ordinary files under a host directory.
• LoopBlockBackend for unprivileged raw-layout testing over fixed-size image files.
• RawBlockBackend for real block devices and partitions using ArgosFS superblocks, labels, allocator state, journal, metadata checkpoints, and data extents directly on the device.

HostFsBackend is not the recommended CapOS rootfs backend. CapOS rootfs support uses loop/raw block semantics and an initramfs flow that mounts ArgosFS at /sysroot before switch_root.

Quick loop-backed rootfs smoke:

cargo build
target/debug/argosfs mkfs --backend loop --images disk0.img,disk1.img,disk2.img --k 2 --m 1 --pool-name capos-root
target/debug/argosfs import-tree --backend loop --images disk0.img,disk1.img,disk2.img /path/to/root /
target/debug/argosfs scan --backend loop --images disk0.img,disk1.img,disk2.img --json
target/debug/argosfs preflight-root --backend loop --images disk0.img,disk1.img,disk2.img --mode rw

See docs/cli.md, docs/raw-backend.md, docs/capos-rootfs.md, and docs/repository-layout.md for CLI selection/configuration, the raw layout, CapOS boot flow, and repository organization.

ArgosFS is a Rust implementation of a self-driving, erasure-coded Linux filesystem. It provides a real FUSE mount frontend suitable for root filesystem experiments, plus a management CLI, health autopilot, repair tooling, and artifact-generation workflows.

ArgosFS is implemented in Rust for memory safety and future kernel-adjacent reuse. The data plane uses mature crates rather than hand-rolled primitives:

• erase for configurable k+m Reed-Solomon layouts.
• fuser/libfuse3 for the Linux filesystem mount frontend.
• zstd and lz4_flex for transparent block compression.
• chacha20poly1305 and argon2 for built-in authenticated encryption.
• io-uring, O_DIRECT, and mmap-backed read staging for advanced I/O.
• serde_json with copy-on-write replacement for durable metadata commits.

Implemented Features

• Real FUSE mount: lookup, getattr, setattr, mknod, mkdir, unlink, rmdir, rename, link, symlink, readlink, open, read, write, statfs, xattrs, readdir, access, create, and fsync.
• Rootfs-critical inode types: regular files, directories, symlinks, hardlinks, character devices, block devices, FIFOs, and sockets.
• POSIX ACL support through native Linux system.posix_acl_access and system.posix_acl_default xattrs, text ACL CLI helpers, inheritance from directory default ACLs, and daemon-side FUSE permission enforcement.
• NFSv4 ACL support through JSON ACLs stored as system.argosfs.nfs4_acl; NFSv4 allow/deny ACEs are evaluated before POSIX ACLs and mode bits.
• Built-in per-stripe authenticated encryption using Argon2id-derived keys and XChaCha20-Poly1305. Missing or invalid keys fail reads/writes with EACCES.
• Reed-Solomon erasure coding with configurable k+m redundancy; layouts such as 4+2 tolerate two failed shards/disks per stripe.
• Per-shard SHA-256 verification and raw-stripe SHA-256 validation.
• Automatic read reconstruction, deferred self-heal when replacement capacity is unavailable, and fsck/scrub repair when enough online disks exist.
• Dynamic add-disk, drain/remove-disk, and weighted rebalance for host volumes; loop/raw pools share the repair/rebalance data path, with raw device replacement still treated as an operator-verified workflow.
• Heterogeneous disk placement through weighted, tier-aware rendezvous hashing.
• Automatic disk probing for SSD/HDD/NVMe class, measured read/write performance, recommended tier, recommended weight, real capacity, and backing block device when sysfs exposes it.
• Hard capacity enforcement: disks with insufficient free ArgosFS shard capacity are excluded from placement, and shard writes fail with ENOSPC before exceeding the recorded capacity.
• Real SMART refresh through smartctl -j -a when a backing block device is available; SMART counters feed the same health and autopilot risk model.
• Dynamic I/O latency feedback: shard reads/writes update per-disk latency EWMA and observed throughput, and placement penalizes slow disks automatically.
• Advanced I/O policy controls: buffered I/O, O_DIRECT with safe buffered fallback for unaligned requests, io_uring with buffered fallback when the kernel denies setup, mmap-backed read staging that still returns owned Vec<u8> buffers, FUSE direct-I/O open flags, and NUMA-aware placement preference when sysfs exposes disk node locality.
• Transparent per-stripe compression with zstd, lz4, or none.
• Built-in Prometheus exporter at /metrics.
• Persistent metadata with copy-on-write JSON commits, triple metadata copies (meta.json, meta.primary.json, and meta.secondary.json), hash-checked transaction journal snapshots, automatic journal replay, double-write mismatch detection, and named metadata snapshots.
• Bounded raw/loop group commit for rootfs workloads: configurable time and transaction limits, data-before-journal ordering, immediate fsync/O_SYNC durability, periodic FUSE commits, and crash-safe deferred extent reclamation.
• RAM + persistent L2 block cache.
• Health scoring from SMART-like counters plus an autopilot planner with persistent risk memory, confirmation/cooldown gates, incremental scrub, policy-separated user intent, background I/O throttling, budgeted rebalance, and metadata-conflict protection.
• Root filesystem integration assets for initramfs and systemd.
• Comprehensive Rust tests and validation workflows for reproducible artifacts.

Build

ArgosFS requires Rust 1.87 or newer. The FUSE frontend also needs libfuse3 headers and pkg-config at build time.

Install the required system packages:

sudo apt-get install -y build-essential pkg-config libfuse3-dev fuse3 smartmontools
curl https://sh.rustup.rs -sSf | sh -s -- -y --profile minimal
. "$HOME/.cargo/env"
rustup default stable

Build and test:

cargo build
cargo test

Development Quick Start (Host Backend)

The host backend is intended for development and compatibility testing. Create a 4+2 volume:

cargo run -- mkfs /var/lib/argosfs/root --disks 6 --k 4 --m 2 --compression zstd

Import a root tree:

sudo cargo run -- import-tree /var/lib/argosfs/root /path/to/rootfs /

Mount it:

sudo cargo run -- mount /var/lib/argosfs/root /mnt/argos-root --foreground

Use it like a normal filesystem:

sudo chroot /mnt/argos-root /bin/sh

Run repair and health automation:

cargo run -- fsck /var/lib/argosfs/root --repair --remove-orphans
cargo run -- health /var/lib/argosfs/root --json
cargo run -- autopilot /var/lib/argosfs/root --once

CLI

Block-backed commands can infer the backend from --images or --devices, and can reuse a JSON device selector with --pool-config FILE. Conflicting selectors are rejected instead of ignored. Size options accept values such as 64MiB and 2GiB; --json is global. See docs/cli.md for the selection, precedence, and output contract.

argosfs mkfs ROOT --disks 6 --k 4 --m 2 --compression zstd
argosfs mount ROOT MOUNTPOINT --foreground -o allow_other
argosfs import-tree ROOT SOURCE_DIR /
argosfs export-tree ROOT DEST_DIR
argosfs put ROOT LOCAL_FILE /path/in/fs
argosfs get ROOT /path/in/fs LOCAL_FILE
argosfs cat ROOT /path/in/fs
argosfs mkdir ROOT /dir --mode 755
argosfs mknod ROOT /dev/null --mode 020666 --rdev 259
argosfs symlink ROOT /target /link
argosfs rename ROOT /old /new
argosfs chmod ROOT /path 644
argosfs truncate ROOT /path 0
argosfs add-disk ROOT --path /mnt/nvme0 --rebalance
argosfs add-disk ROOT --tier hot --weight 2.0 --capacity-bytes 1000000000000 --rebalance
argosfs probe-disks ROOT
argosfs refresh-smart ROOT
argosfs remove-disk ROOT disk-0003
argosfs mark-disk ROOT disk-0002 failed
argosfs set-health ROOT disk-0001 --pending-sectors 12 --latency-ms 140
argosfs fsck ROOT --repair --remove-orphans
argosfs scrub ROOT
argosfs rebalance ROOT
argosfs autopilot ROOT --interval 60
argosfs autopilot ROOT --policy /etc/argosfs/autopilot-policy.json --interval 60
argosfs autopilot ROOT --dry-run --explain --json
argosfs snapshot ROOT before-upgrade
argosfs enable-encryption ROOT --key-file /etc/argosfs.key --reencrypt
argosfs enable-encryption ROOT --passphrase-stdin
argosfs encryption-status ROOT
argosfs set-io-mode ROOT --mode io-uring
argosfs set-io-mode ROOT --mode direct --direct-io
argosfs prometheus ROOT --listen 127.0.0.1:9108
argosfs set-posix-acl ROOT /etc/shadow 'user::rw-,group::---,mask::---,other::---'
argosfs get-posix-acl ROOT /etc/shadow
argosfs set-nfs4-acl ROOT /srv/data @nfs4-acl.json
argosfs get-nfs4-acl ROOT /srv/data
argosfs verify-journal ROOT
argosfs inspect-pool --pool-config /etc/argosfs/root-pool.json
argosfs fsck --pool-config /etc/argosfs/root-pool.json --repair

add-disk defaults to automatic probing. Pass --tier, --weight, or --capacity-bytes only when you want to override the probe result. Modes accept decimal, octal (755 or 0o755), and hex (0x...) syntax.

mount --foreground is accepted for compatibility with service files and scripts. The current libfuse frontend runs in the foreground either way.

Encryption reads the key from ARGOSFS_KEY or ARGOSFS_KEY_FILE during normal operation. Use --key-file or --passphrase-stdin when enabling encryption. --passphrase remains available only for tests because argv can be exposed in shell history, process listings, logs, and crash reports. enable-encryption --reencrypt rewrites existing file stripes so old data becomes encrypted at rest; new writes are encrypted immediately after encryption is enabled. set-io-mode persists the data-plane policy in volume metadata, and --direct-io also asks FUSE clients to use direct I/O handles.

verify-journal validates the transaction hash chain, metadata snapshot hashes, and all metadata copies. Opening a volume also performs recovery: ArgosFS picks the newest valid metadata copy, replays a newer journal snapshot when needed, and rewrites inconsistent metadata copies.

For crash testing, set ARGOSFS_CRASH_POINT to one of before-journal, after-journal, after-primary-metadata, after-secondary-metadata, or after-compatible-metadata. Set ARGOSFS_CRASH_ABORT=1 to abort the process instead of returning an injected I/O error.

Root Filesystem Use

Production-style rootfs use selects a loop/raw pool rather than a host volume. Keep the device list and expected identity in one JSON selector:

sudo install -d -m 0755 /etc/argosfs
sudo install -m 0644 docs/pool-config.example.json \
  /etc/argosfs/root-pool.json
argosfs fsck --pool-config /etc/argosfs/root-pool.json \
  --repair --remove-orphans
argosfs preflight-root --pool-config /etc/argosfs/root-pool.json --mode rw
argosfs mount-root --pool-config /etc/argosfs/root-pool.json \
  --target /newroot --mode rw --foreground

An initramfs must include the binary, libfuse3, /dev/fuse, stable device identification, and the replay/fsck/preflight/mount sequence before switch_root. See docs/rootfs.md, docs/boot.md, docs/capos-rootfs.md, and integrations/ for complete flows. The positional mount ROOT MOUNTPOINT command remains the development-only host-backend frontend.

Validation Artifacts

Run the validation workflow:

python3 scripts/validation/run_full_validation.py --output target/argosfs-artifacts/runs/manual
scripts/experiments/run_all.sh --quick --output target/argosfs-artifacts/runs/ae-quick

The run directory contains:

• deterministic input datasets,
• the generated ArgosFS volume,
• command logs,
• JSON health/fsck/autopilot/rebalance reports,
• CSV timing samples,
• a manifest with build, kernel, and ArgosFS configuration.

See docs/artifact-evaluation.md, docs/compatibility-report.md, docs/performance-report.md, docs/self-driving-model.md, docs/safety-invariants.md, and docs/related-work.md for paper-oriented reproduction and positioning. A static project website lives in website/index.html.