Sampler: implement no-repeat-ngram (hard ban on repeating an n-gram)

[pekkah]

Motivation

Same finding as #464: the penalty family demotes tokens, not sequences, and verbatim reply replay was measured at ~50% incidence on a 12B roleplay model with no penalty active — unmoved by RepetitionPenalty, PenaltyLastN, or PenaltySeedFromPrompt. See #463 for the data.

no_repeat_ngram_size is the blunt instrument in this space, and its bluntness is the point: it is a hard constraint, not a nudge, so it cannot be defeated by a peaked-enough distribution. Where DRY (#464) scales a penalty by match length, this simply makes the repeat unreachable.

What it is

HuggingFace transformers' no_repeat_ngram_size / llama.cpp's --no-repeat-ngram-size. Given size n: look at the last n-1 emitted tokens, find every place that same (n-1)-gram occurred earlier in the context, and set the logit of each token that followed it to -inf. The model can never emit an n-gram it has already emitted.

One parameter, no_repeat_ngram_size (0 = off; 3–5 typical).

Why implement it alongside DRY rather than instead

They fail differently and are worth having both:

• no-repeat-ngram is absolute and cheap, but it bans legitimate repetition too — proper nouns, code identifiers, formatting runs, "said". On code or structured output a small n is actively harmful. It also cannot express "a bit of repetition is fine, a lot is not."
• DRY is graded and tunable and leaves short repeats alone, but a sufficiently confident model can still pay the penalty and repeat anyway.

Small n catches loops early; DRY handles the long tail. Different users want different tradeoffs, and both are standard enough that clients pass them.

Implementation notes

• Needs the ordered token history; PenaltyWindow (IReadOnlyList<int>, oldest-first) already provides it.
• Both sampling paths — Sampler.Sample's slow path and the inlined block in SampleTopK, which carry lockstep warnings from fix(server): expose repetition_penalty and penalty_last_n per request #461.
• A ban is an extreme demotion, so SampleTopK's over-select reasoning holds and promotingPenalty (Sampler.cs:40-41) needs no new case.
• Watch the degenerate case: banning can empty the candidate set (short n, long constrained generation, or combined with a grammar constraint). SampleTopK already has an all--inf guard that falls back to Greedy; the slow path's Normalize/SampleFromDistribution would need checking, and the intended semantics decided — llama.cpp lets the ban win and can stall. Fallback behaviour should be explicit and tested, not emergent.
• Interaction with ITokenConstraint (grammar / JSON-schema decoding) needs a decision: two independent maskers can jointly empty the support. Constraint should probably win.
• Cost is O(context) per token with a rolling scan, or better with a hash of (n-1)-gram → follower set built incrementally. Hot path is allocation-free — the index must be a reusable structure, rebuilt per request, not per token.
• Surfaces per the fix(server): expose repetition_penalty and penalty_last_n per request #461/feat(server): per-request top_k and min_p #462 convention: SamplingParams, CLI flag, per-request field on both HTTP surfaces.

Tests

tests/SharpInference.Tests.ForwardPass, model-free: synthetic history containing a known n-gram, assert the follower token's post-filter probability is exactly 0 and that an unrelated token is untouched; assert n=0 is byte-identical to no sampler at all; assert the empty-support fallback does something defined.

References

• HF NoRepeatNGramLogitsProcessor
• llama.cpp --no-repeat-ngram-size