|
| 1 | +# ADR 0022: Extract FSST into a standalone module, ported faithfully from the paper |
| 2 | + |
| 3 | +- **Status:** Accepted |
| 4 | +- **Date:** 2026-07-20 |
| 5 | +- **Deciders:** project maintainer |
| 6 | +- **Supersedes:** — |
| 7 | +- **Superseded by:** — |
| 8 | +- **Related:** [ADR 0005 — Vector API adoption](0005-vector-api-adoption.md), [ADR 0010 — Lazy |
| 9 | + decode](0010-lazy-decode.md), [ADR 0017 — In-house FlatBuffers codegen](0017-in-house-flatbuffers-codegen.md) |
| 10 | + |
| 11 | +## Context |
| 12 | + |
| 13 | +`writer/FsstEncodingEncoder` and `reader/FsstEncodingDecoder` held a first-pass, simplified FSST |
| 14 | +(Fast Static Symbol Table) implementation. Comparing it against `vortex-jni` (issue #287) surfaced |
| 15 | +real gaps against both the original paper (Boncz/Neumann/Leis, "FSST: Fast Random Access String |
| 16 | +Compression", PVLDB 13(11), 2020) and the Rust reference (`spiraldb/fsst`, the crate the Rust |
| 17 | +`vortex-fsst` wire adapter is built on): |
| 18 | + |
| 19 | +- **Matching.** The old `longestMatch` probed symbol lengths 8 down to 1 in a per-position loop — |
| 20 | + up to eight sequential open-addressing hash lookups per input byte, a variable-length loop body |
| 21 | + that blocks JIT auto-vectorization (the exact shape CLAUDE.md's hot-loop rule forbids). The |
| 22 | + paper's Algorithm 4 ("lossy perfect hashing", §5.1) resolves any match in O(1): a direct |
| 23 | + 65536-entry array for 0/1/2-byte symbols, and a small hash table for 3-8 byte symbols, combined |
| 24 | + branch-free. |
| 25 | +- **Training.** The old bottom-up loop always compress-counted the *full* training sample on every |
| 26 | + one of its five generations, ranked candidates by plain `count * length`, and never pruned |
| 27 | + low-frequency noise. The real reference (`spiraldb/fsst`) additionally uses a growing |
| 28 | + per-generation sample fraction (~6% → 100%), a per-generation min-count floor, an 8x gain boost |
| 29 | + for single-byte candidates (to suppress escapes), and a final cost-based prune pass. |
| 30 | +- **Decode.** The old decoder copied a matched symbol's bytes one at a time in a per-byte loop |
| 31 | + instead of the paper's Algorithm 1 trick — one unconditional 8-byte store, advancing the output |
| 32 | + cursor by only the symbol's true length. |
| 33 | + |
| 34 | +A zero-dependency Java FSST implementation was searched for and does not exist (only the C |
| 35 | +reference `cwida/fsst`, the Rust `spiraldb/fsst`, and a Go port `axiomhq/fsst`). Per that search's |
| 36 | +own conclusion, the fix isolates FSST into its own module — mirroring how the Rust reference itself |
| 37 | +splits a pure algorithm crate (`spiraldb/fsst`, zero dependencies, no Vortex awareness) from |
| 38 | +vortex-rust's thin wire adapter (`vortex-fsst`) — so the compression algorithm can be iterated on |
| 39 | +and benchmarked without the Vortex wire-format "shell" in the way. |
| 40 | + |
| 41 | +During the rewrite's own review, a real data-corruption bug was found and fixed: the new |
| 42 | +branch-free matcher has no notion of "end of input" (unlike the old bounded loop), so a trained |
| 43 | +symbol whose trailing bytes happen to be zero could spuriously match zero-padded bytes past a short |
| 44 | +input's true end, silently appending garbage bytes to the decoded output. Confirmed with a concrete |
| 45 | +repro (a 3-byte input round-tripping to 8 bytes) before it reached a merged PR. Fixed by an explicit |
| 46 | +`pos + length <= end` guard at both call sites that feed the matcher a possibly-short remaining |
| 47 | +range (`Compressor.compress` and the training loop's compress-count step) — the paper's own |
| 48 | +guidance for a branch-free scalar kernel (§5.2) names exactly this bound as the alternative to a |
| 49 | +terminator byte (which is an AVX512-batch-kernel-specific optimization, out of scope here). |
| 50 | + |
| 51 | +## Decision |
| 52 | + |
| 53 | +Extract FSST into a new top-level Maven module, `fsst` (artifact `vortex-fsst`), with zero |
| 54 | +dependency on `core`/`reader`/`writer`: |
| 55 | + |
| 56 | +1. **Port the paper faithfully**, plus the real reference's engineering refinements: branch-free |
| 57 | + O(1) matching (`ShortCodeTable` for 0/1/2-byte symbols, a 2048-slot lossy perfect hash table for |
| 58 | + 3-8 byte symbols — matching `spiraldb/fsst`'s L1D-cache-line-split sizing over the paper's |
| 59 | + literal 4096, since that Rust crate is the actual `vortex-jni` comparison target); adaptive |
| 60 | + byte-size-bounded sampling with a growing per-generation fraction, min-count pruning, the |
| 61 | + single-byte gain boost, and the final cost-based prune during training; the unconditional-8-byte- |
| 62 | + store decode trick. |
| 63 | +2. **Memory boundary: `MemorySegment`, not `ByteBuffer`.** `java.lang.foreign` is a standard JDK |
| 64 | + module, not a third-party dependency — using it satisfies "zero deps" exactly as well as |
| 65 | + `ByteBuffer` would, while staying consistent with every other module in this codebase (CLAUDE.md: |
| 66 | + "Uses FFM (`MemorySegment`/`Arena`) — never JNI or `sun.misc.Unsafe`") and avoiding a dual-API |
| 67 | + translation layer with its own parity-testing burden. The module's hot-path `compress`/ |
| 68 | + `decompress` methods take `MemorySegment` directly; plain `byte[]` overloads exist alongside them |
| 69 | + for standalone use with zero FFM ceremony. |
| 70 | +3. **`writer`/`reader` become thin wire adapters.** `FsstEncodingEncoder`/`FsstEncodingDecoder` keep |
| 71 | + all Vortex-specific plumbing (UTF-8 conversion, `ProtoFSSTMetadata`, `EncodeNode`/`EncodeResult` |
| 72 | + assembly, `Arena` allocation) and delegate the algorithm itself to `CompressorBuilder`/ |
| 73 | + `Compressor`/`Decompressor`. The wire format (`vortex.fsst`) is unchanged — this is a pure |
| 74 | + algorithm/performance rewrite, not a wire-format change. The one adapter-side subtlety: the |
| 75 | + module numbers codes gain-descending internally (load-bearing for the hash table's |
| 76 | + first-writer-wins collision rule), while the wire format requires length-sorted order |
| 77 | + (`Compressor#codesSortedByLength()`), so the writer remaps every code the compressor emits |
| 78 | + through the wire permutation before writing it out. |
| 79 | + |
| 80 | +## Consequences |
| 81 | + |
| 82 | +### Positive |
| 83 | + |
| 84 | +- Measured, real speedup (`JavaVsJniFsstBenchmark`, unmodified before and after — see the |
| 85 | + `[Unreleased]` CHANGELOG entry for the full table): `javaFsstEncode` 0.085 → 1.848 ops/s (~21.7x |
| 86 | + faster, closing the gap to `vortex-jni` from 36x slower to 1.6x slower); `javaFsstDecode` 5.243 → |
| 87 | + 27.137 ops/s (~5.2x faster, gap 6.5x → 1.3x). |
| 88 | +- The `fsst` module is independently testable and benchmarkable with zero Vortex file-format |
| 89 | + scaffolding in the loop (`FsstEncodingEncoderTest`'s wire-format tests are unaffected; new |
| 90 | + `fsst`-module tests exercise the algorithm directly) — the issue's own stated motivation for the |
| 91 | + extraction. |
| 92 | +- A golden test (`PaperFigure2Test`) pins the paper's own worked example against a hand-built |
| 93 | + (not trained) symbol table — an external, human-verifiable fixed point independent of this |
| 94 | + project's training heuristics. |
| 95 | + |
| 96 | +### Negative |
| 97 | + |
| 98 | +- Real rework: five new PRs' worth of module surface (`Symbol`, `Matcher`/`ShortCodeTable`/ |
| 99 | + `LossyPerfectHashTable`, `Sample`/`TrainingGeneration`/`CompressorBuilder`/`Compressor`, |
| 100 | + `Decompressor`) versus the ~250-line inline implementation it replaces. |
| 101 | +- The compressor's internal (gain-descending) code numbering versus the wire's (length-sorted) |
| 102 | + numbering is a second permutation the adapter must get exactly right in both directions (symbol |
| 103 | + table population and code-stream remapping) — a subtle class of bug (wrong output, not a crash) |
| 104 | + that needs its own dedicated attention in any future change to either ordering. |
| 105 | + |
| 106 | +### Risks to manage |
| 107 | + |
| 108 | +- The branch-free matcher's lack of an input-end bound is a **structural** property, not a bug that |
| 109 | + was simply fixed once — any future caller of `Matcher.longestMatch`/`Compressor.compress` that |
| 110 | + operates on a range shorter than 8 bytes from the true end of a real buffer must carry the same |
| 111 | + `pos + length <= end` bound. This is exactly the kind of thing a future port or refactor could |
| 112 | + silently drop, since it is not obviously part of the "core algorithm." |
| 113 | +- `LossyPerfectHashTable`'s 2048-slot sizing is a deliberately chosen constant (matching the |
| 114 | + `spiraldb/fsst` reference), not derived from first principles for this JVM's cache behavior — if a |
| 115 | + future benchmark shows a different size wins on the actual hardware/workload this project cares |
| 116 | + about, revisit it deliberately rather than assuming 2048 is universally correct. |
| 117 | + |
| 118 | +## Alternatives considered |
| 119 | + |
| 120 | +- **Reuse an existing Java library.** Searched (GitHub, web) and found none with zero dependencies; |
| 121 | + the only real Java-ecosystem candidates were the C reference (via JNI, which this project's FFM |
| 122 | + mandate rules out) and ports in other languages entirely (Rust, Go). |
| 123 | +- **Keep the old inline implementation and just tune constants.** Rejected: the old matching |
| 124 | + algorithm's O(8)-probes-per-byte structure is the dominant cost, not a constant that tuning could |
| 125 | + fix — no amount of tuning turns a sequential 8-probe loop into an O(1) branch-free lookup. |
| 126 | +- **`ByteBuffer` instead of `MemorySegment` at the module boundary.** Rejected: reintroduces a dual |
| 127 | + API (parity tests, two hot-path implementations to keep in sync) for no benefit over |
| 128 | + `java.lang.foreign`, which is already a zero-dependency JDK-standard API and is what every other |
| 129 | + module in this codebase uses for the same purpose. |
| 130 | + |
| 131 | +## References |
| 132 | + |
| 133 | +- The paper: Boncz, Neumann, Leis, "FSST: Fast Random Access String Compression", PVLDB 13(11), |
| 134 | + 2020. <https://www.vldb.org/pvldb/vol13/p2649-boncz.pdf> |
| 135 | +- Rust reference: [`spiraldb/fsst`](https://github.com/spiraldb/fsst) on GitHub. |
| 136 | +- The 8-PR sequence (all on `main`): |
| 137 | + - `2b8db4f2` — module skeleton, `Symbol`, `Decompressor` |
| 138 | + - `8d778bac` — branch-free matching (`ShortCodeTable` + `LossyPerfectHashTable` + `Matcher`) |
| 139 | + - `1003a673` — `JavaVsJniFsstBenchmark` baseline (pre-rewrite numbers) |
| 140 | + - `4158c924` — training (adaptive sampling, min-count pruning, gain boost, final prune) — includes |
| 141 | + the boundary-overrun fix described in Context |
| 142 | + - `76e26eb6` — `MemorySegment` hot paths + unconditional-store decode |
| 143 | + - `1b9714c7` — rewire `writer`/`reader` adapters onto the `fsst` module |
| 144 | + - `75f58734` — golden test for the paper's Figure 2 worked example |
| 145 | + - `b218740f` — CHANGELOG before/after benchmark table |
0 commit comments