splitbench

Here is 1 public repository matching this topic...

ahnaf-lab / splitbench

A zero-dependency Node library and CLI harness that runs several small tokenizer implementations (whitespace, naive BPE, char-level, toy unigram) over the same text and benchmarks token count, word…

nodejs javascript cli automation claude-code splitbench

Updated Nov 1, 2026
JavaScript

Add this topic to your repo

To associate your repository with the splitbench topic, visit your repo's landing page and select "manage topics."

Learn more