Skip to content

Cap kMaxArcs by Unsigned in CompactArcStore and ConstFstImpl. - #351

Merged
copybara-service[bot] merged 1 commit into
mainfrom
copybara/995274956
Oct 8, 2026
Merged

copybara-service[bot] merged 1 commit into
mainfrom
copybara/995274956

Conversation

@copybara-service

Copy link
Copy Markdown

Cap kMaxArcs by Unsigned in CompactArcStore and ConstFstImpl.

CompactArcStore<Element, Unsigned> and ConstFstImpl<Arc, Unsigned> can be
instantiated with smaller unsigned index types such as uint8_t and uint16_t
(compact8_*, compact16_*, const8, const16), where state offsets, arc
counts, and epsilon counts are stored in Unsigned. Previously:

  1. kMaxArcs was hardcoded to 1ULL << 52 regardless of Unsigned, so
    Read() did not reject headers or state-offset tables whose arc/compact
    counts exceeded std::numeric_limits<Unsigned>::max().
  2. The constructors did not check ncompacts_ > kMaxArcs or narcs_ > kMaxArcs
    before truncating offsets/counts into Unsigned, silently wrapping around
    for inputs exceeding the index type's capacity.
  3. In CompactFstImpl(const Fst<Arc>&, ...), if (compactor_->Error()) called
    SetProperties(kError, kError) without returning, allowing the subsequent
    1-argument SetProperties(...) call at the end of the constructor to
    overwrite kError.

Fix by:

  • Capping kMaxArcs on CompactArcStore and ConstFstImpl at
    std::min<uint64_t>(0x10000000000000ull, std::numeric_limits<Unsigned>::max()).
  • Checking ncompacts_ > kMaxArcs in CompactArcStore's FST and iterator
    constructors, and checking data->nstates_ > kMaxArcs / arc_compactor.Size()
    in the fixed out-degree branch of CompactArcStore::Read.
  • Returning early in CompactFstImpl(const Fst<Arc>&, ...) when
    compactor_->Error() is true.
  • Checking narcs_ > kMaxArcs in ConstFstImpl(const Fst<Arc>&).
  • Adding unit tests in fst_limits_test for uint8_t index overflow and
    64-bit kMaxArcs bounds.

`CompactArcStore<Element, Unsigned>` and `ConstFstImpl<Arc, Unsigned>` can be
instantiated with smaller unsigned index types such as `uint8_t` and `uint16_t`
(`compact8_*`, `compact16_*`, `const8`, `const16`), where state offsets, arc
counts, and epsilon counts are stored in `Unsigned`. Previously:

1. `kMaxArcs` was hardcoded to `1ULL << 52` regardless of `Unsigned`, so
   `Read()` did not reject headers or state-offset tables whose arc/compact
   counts exceeded `std::numeric_limits<Unsigned>::max()`.
2. The constructors did not check `ncompacts_ > kMaxArcs` or `narcs_ > kMaxArcs`
   before truncating offsets/counts into `Unsigned`, silently wrapping around
   for inputs exceeding the index type's capacity.
3. In `CompactFstImpl(const Fst<Arc>&, ...)`, `if (compactor_->Error())` called
   `SetProperties(kError, kError)` without returning, allowing the subsequent
   1-argument `SetProperties(...)` call at the end of the constructor to
   overwrite `kError`.

Fix by:
- Capping `kMaxArcs` on `CompactArcStore` and `ConstFstImpl` at
  `std::min<uint64_t>(0x10000000000000ull, std::numeric_limits<Unsigned>::max())`.
- Checking `ncompacts_ > kMaxArcs` in `CompactArcStore`'s FST and iterator
  constructors, and checking `data->nstates_ > kMaxArcs / arc_compactor.Size()`
  in the fixed out-degree branch of `CompactArcStore::Read`.
- Returning early in `CompactFstImpl(const Fst<Arc>&, ...)` when
  `compactor_->Error()` is true.
- Checking `narcs_ > kMaxArcs` in `ConstFstImpl(const Fst<Arc>&)`.
- Adding unit tests in `fst_limits_test` for `uint8_t` index overflow and
  64-bit `kMaxArcs` bounds.

PiperOrigin-RevId: 995469267
@copybara-service
copybara-service Bot merged commit 7f3847b into main Oct 8, 2026
2 checks passed
@copybara-service
copybara-service Bot deleted the copybara/995274956 branch October 8, 2026 00:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant