Sampler: implement DRY (Don't Repeat Yourself) sequence-repetition penalty

[pekkah]

Motivation

Measured during the investigation that closed with #463: verbatim reply replay on a 12B roleplay model repeating an identical user message occurred in roughly half of conversations with no penalty active at all, and neither raising RepetitionPenalty, widening PenaltyLastN, nor disabling PenaltySeedFromPrompt moved it (21 conversations, N=3–4 per arm).

That is the expected result, not a surprise: the penalty family demotes tokens, not sequences. A token-level penalty cannot distinguish "this word appeared recently" from "we are 40 tokens into re-emitting a previous reply verbatim" — and the second is the failure users actually notice. FrequencyPenalty (#459) scales with occurrence count but still has no notion of position or contiguity.

DRY is the sampler designed for exactly this.

What it is

Introduced by @pi6am for KoboldCpp, upstreamed into llama.cpp (--dry-multiplier). For each candidate token, DRY finds the longest suffix of the current context that would be extended by emitting it and that has occurred before, then applies a penalty growing exponentially in that match length:

penalty = multiplier * base ^ (match_length - allowed_length)   // when match_length >= allowed_length

Subtracted from the logit. The effect: continuing a short repeat is nearly free, continuing a long verbatim repeat is prohibitive. It attacks replay while leaving ordinary word reuse alone — the specific thing RepetitionPenalty cannot do.

Parameters (llama.cpp names): dry_multiplier (0 = off), dry_base (default 1.75), dry_allowed_length (default 2), dry_penalty_last_n, and sequence breakers — token strings (\n, :, ", *) that reset matching so it does not chain across turn or dialogue boundaries.

Implementation notes

• Needs the ordered token history, not the distinct-token set DistinctTokens builds. PenaltyWindow already is an ordered ring (IReadOnlyList<int>, oldest-first), so it can serve as-is.
• Must land in both sampling paths — Sampler.Sample's slow path and the inlined block in SampleTopK — which are duplicated by design and already carry lockstep warnings from fix(server): expose repetition_penalty and penalty_last_n per request #461.
• DRY only ever demotes, so SampleTopK's over-select argument holds unchanged and promotingPenalty gating (Sampler.cs:40-41) does not need to grow a case.
• The naive algorithm is O(context²) per token. llama.cpp uses Z-algorithm to get it to O(context). The sampler hot path is allocation-free (NativeMemory/Span<T>, no LINQ/closures/boxing per token) — the match table needs to be a reusable buffer, not a per-token allocation. This is the main design risk and worth prototyping before committing to a shape.
• Surfaces, following the convention established by fix(server): expose repetition_penalty and penalty_last_n per request #461/feat(server): per-request top_k and min_p #462: SamplingParams fields, CLI flags mirroring llama.cpp's names, per-request fields on the OpenAI and Anthropic surfaces.
• Sequence breakers are string→token-id work at request setup, not per token.

Tests

tests/SharpInference.Tests.ForwardPass. The property is directly assertable without a model: build a logit vector plus a synthetic history containing a repeated span, and assert the token that would extend the longest match is demoted in proportion to match length, and that a sequence breaker resets it. Slow/fast path agreement should be pinned the same way PenaltyPathEquivalenceTests pins it for the existing penalties.

References

• llama.cpp llama_sampler_dry — src/llama-sampling.cpp
• Original proposal: added implementation of DRY sampler ggml-org/llama.cpp#6839