Skip to content

npu tokenizer: read a vocabulary whose normalizer rewrites spaces to the word-boundary symbol and prepends none (fixes #2215) - #2217

Merged
bong-water-water-bong merged 3 commits into
mainfrom
cl-npuperf/tokenizer-replace-meta
Nov 11, 2026
Merged

bong-water-water-bong merged 3 commits into
mainfrom
cl-npuperf/tokenizer-replace-meta

Commits

  1. Commits on Nov 11, 2026