Skip to content

Identify and extract SPIFFS filesystems (#17) - #39

Merged
nmatt0 merged 1 commit into
masterfrom
feat/spiffs
Sep 18, 2026
Merged

nmatt0 merged 1 commit into
masterfrom
feat/spiffs

Conversation

@nmatt0

@nmatt0 nmatt0 commented Sep 17, 2026

Copy link
Copy Markdown
Owner

Summary

SPIFFS is the classic SPI-NOR filesystem on ESP8266 / ESP32-classic and other small MCUs, and neither binwalk nor unblob extract it — web assets, config, and credentials commonly live in a SPIFFS partition. This adds identification and byte-exact extraction.

SPIFFS has no superblock magic, and its page/block geometry is build-time config that is not stored in the image, so both are handled specially.

What it does

  • Identify — anchors on a committed object-index header: at a page boundary it is span_ix 0, flags 0xF8 (USED|FINAL|INDEX cleared), then 3 align zero-bytes, giving the 6-byte pattern 00 00 F8 00 00 00 (once per file). The validator then infers the geometry over the image and confirms a coherent object graph, so false anchors are rejected. It reports the inferred page/block sizes.
  • Geometry inference — try candidate (page, block) sizes and score by how many files reassemble completely, then bytes, then file count. The correct geometry recovers full files; a wrong one finds index headers at aligned offsets but truncates the data, so completeness disambiguates.
  • Extract — reassemble each object from its FINAL index header (name + size) and its data pages ordered by span index, into the existing SafeRoot. A 64 MiB guard bounds inference cost against a stray anchor.

Scoped to a standalone SPIFFS image — a dumped partition, or one moria extracts and re-scans (the dominant workflow). An embedded-at-offset SPIFFS is reached via partition extraction.

Provenance

The page/object layout is a clean reimplementation of the SPIFFS on-disk format (MIT), written against moria's Reader and verified byte-exact against real mkspiffs images. Noted in the README's license section.

Tests

  • A minimal-image fixture in gen_samples (identifies at the consistent tier).
  • tests/test_spiffs.py: synthetic identify + extract, a false-positive guard (a bare anchor with no object graph is rejected), and a real mkspiffs round-trip that extracts every file byte-exact across two geometries (page 256/block 4096 and page 512/block 8192), self-skipping when the tool is absent.
  • Validated on real images spanning small files through a multi-block file, nested directories, and both geometries. False-positive-clean across a large firmware corpus; ASan clean; libFuzzer smoke over the parser with no crashes.

Closes #17

SPIFFS is the classic SPI-NOR filesystem on ESP8266 / ESP32-classic and other
small MCUs, and neither binwalk nor unblob extract it. Add identification and
byte-exact extraction.

SPIFFS has no superblock magic and its page/block geometry is build-time config
that is not stored in the image, so both are handled specially:

- Identify: anchor on a committed object-index header (at a page boundary it is
  span_ix 0, flags 0xF8, then 3 align zero-bytes -> the 6-byte pattern
  00 00 F8 00 00 00, once per file). The validator infers the geometry over the
  image and confirms a coherent object graph, so false anchors are rejected.
- Geometry inference: try candidate (page, block) sizes and score by how many
  files reassemble completely, then bytes, then file count. The correct geometry
  recovers full files; a wrong one finds index headers at aligned offsets but
  truncates the data, so completeness disambiguates.
- Extract: reassemble each object from its FINAL index header (name + size) and
  its data pages ordered by span index, into the SafeRoot. Scoped to a standalone
  SPIFFS image (a dumped partition, or one moria extracted and re-scanned); a
  64 MiB guard bounds inference cost against a stray anchor.

The page/object layout is a clean reimplementation of the SPIFFS on-disk format
(MIT), verified byte-exact against real mkspiffs images (noted in README).

- New: signatures/spiffs.toml, src/spiffs_parse.{hpp,cpp} (shared core),
  src/validators/spiffs.{hpp,cpp}, src/extract/spiffs.{hpp,cpp}. Wired into the
  validator registry, extractor registry, MIME map and build.
- Tests: a minimal-image fixture in gen_samples, and tests/test_spiffs.py
  (synthetic identify/extract, a false-positive guard on a bare anchor, and a
  real mkspiffs round-trip that extracts every file byte-exact for two geometries,
  self-skipping when the tool is absent). Wired into run.sh and CTest.
@nmatt0
nmatt0 merged commit 02aca93 into master Sep 18, 2026
4 checks passed
@nmatt0
nmatt0 deleted the feat/spiffs branch September 18, 2026 00:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SPIFFS: identify and extract

1 participant