server : preserve RAM-cached branches sharing a large prompt prefix

[arc-uri-el]

Overview

Independent conversations can share most of a system message or tool schema while differing in a short suffix. The LCP slot-selection path only saves and consults the RAM prompt cache when fewer than half of the current tokens would survive. Consequently, two such conversations overwrite one another's suffix state without retaining it, even with --cache-ram enabled.

Use actual suffix replacement instead of the half-prefix-loss threshold. The existing RAM cache remains the only storage owner and keeps its existing capacity limit. Pure prefix extension does not enter the new path. No new cache, option, checkpoint API or scheduling policy is added.

The existing slot test now alternates two 241-token prompts with a 193-token common prefix. Before the change, returning to either branch reuses 193 tokens and evaluates 48. Afterward it restores 240 and evaluates one. The test compares generated tokens against the same one-token replay shape before displacement, rather than comparing a one-token decode to a full-prefill numerical path.

Measurements

Device:     Ryzen AI MAX+ 395 / ASUS ROG Flow Z13 GZ302EAC
Memory:     128 GB LPDDR5X
Power:      AC; performance platform profile and CPU governor; GPU clocks auto
Firmware:   BIOS 304; PPT settings 80/92/93 W; VRAM 512 MiB
Kernel:     Linux 7.3.0-rc1; ttm.pages_limit=30408704; GTT 116 GiB
Backend:    CPU and Radeon 8060S Vulkan RADV, Mesa 26.2.0
Build:      GCC 15.2, Release, shared libraries, native CPU, Vulkan, OpenMP; -j8
Baseline:   7449a0fe9710ab584c5f9a6d25e7a31eea2708b8
Change:     86d020f99cf0b9f918e421044191c60017d30a5a (runtime change)
Test:       9cbefdcf068ca21cf57d81cc153bdcede67ea510 (replay control refinement only)
Model:      ggml-org/test-model-stories260K; official Qwen3.8-27B Q4_K_M

Baseline and candidate were built separately in this session with identical settings, with no compiler warnings. The test-only follow-up does not change the measured executable. Attribution-only commit updates preserve the exact tested source trees. The later master merge 99a40a3e6 changes speculative replay at a different site; this witness uses target-only generation.

Baseline / after:

Scenario	Baseline cached / fresh	Candidate cached / fresh
Tiny model, branch A after B	193 / 48	240 / 1
Tiny model, branch B after A	193 / 48	240 / 1
Official Qwen, tool-result continuation A	993 / 82	1049 / 26
Official Qwen, tool-result continuation B	993 / 82	1049 / 26

The submitted tiny-model regression fails its cache assertion on the baseline and passes on the candidate on both CPU and Vulkan. Both restored branches produce the exact token IDs of their undisplaced, same-shape replay controls. The fixture is the existing ggml-org/test-model-stories260K preset, file stories260K-f32.gguf, SHA-256 270cba1bd5109f42d03350f60406024560464db173c0e387d91f0426d3bd256d.

The official-Qwen witness uses one 131,072-context slot, F16 KV, batch 2048/512, no speculative decoding, and a common system prefix plus the same record_value tool schema. Conversation A requests value 73 and B requests 91. Both runtimes make the right native calls and return their own exact receipts after interleaving. The candidate's two prompt phases take 482/481 ms versus baseline 755/751 ms. Those two observations are not a throughput claim or a variance estimate; the directly asserted improvement is 82 fresh tokens reduced to 26.

Correctness:

Run the focused test through the existing server test suite after its normal preset setup:

LLAMA_SERVER_BIN_PATH=/absolute/path/to/build/bin/llama-server \
  python -m pytest tools/server/tests/unit/test_slot_save.py \
  -k ram_cache_interleaved_shared_prefix

Local execution used that exact test function and the normal ServerProcess helper, selecting the already-downloaded preset file explicitly to avoid downloading unrelated presets. Baseline and candidate library mappings and actual CPU/Vulkan KV placement were checked. All test processes exited and were verified absent.

An initial synthetic-fixture check compared full-prefill output with one-token replay and found different IDs. A baseline-only undisplaced replay reproduced the same difference. The test now holds replay shape constant before and after displacement; no tolerance or cache assertion was weakened. A synthetic architecture fixture without a tokenizer was also rejected as a text-server witness and was not counted as acceptance.

llama-bench, perplexity and the full backend-op matrix were not run. No math kernel is changed and no general decode speedup is claimed. Additional branch saves consume the existing cache budget and copy time; the regression targets recovery of overwritten suffixes, not zero-overhead caching.

Requirements

• I have read and agree with the contributing guidelines.
• This change is justified by the actual Strix Halo interleaved-conversation measurements above.
• AI usage disclosure: AGENT-AUTHORED. GPT 6 Astra isolated the existing local fix, wrote and executed the regression and controls, and prepared this PR at the explicit direction of the submitting account's owner. The submitting account owns review and follow-up.
• What was NOT verified: other GPU backends, other operating systems, full occupied contexts, broad concurrency and speculative-decoding matrices, perplexity or a general performance matrix. This is targeted prompt-cache coverage, not global serving qualification.

[arc-uri-el]

The CANN notification is a pre-existing workflow validation failure, not a compiler or test failure from this PR.

Run 34227573493 reports:

Invalid workflow file: .github/workflows/build-cann.yml
(Line: 37, Col: 6): Unexpected value ''

jobs: at line 37 has no entries because the entire CANN job is commented out. The REST API reports zero jobs and zero check runs, so no CANN build was attempted. The workflow blob is identical in this PR, its tested base 7449a0fe9, and master 99a40a3e6 (8b9208c3875917319b17930f64b837570371b8a2). The master push run for 99a40a3e6 reports the exact same line-37 error.

The Server workflow is a separate state: GitHub says it is awaiting approval from a maintainer. It has not run yet. The local CPU/Vulkan regression evidence in the PR remains applicable; it is not being represented as CANN coverage.

I have kept unrelated workflow changes out of this focused cache fix. The disabled CANN workflow definition needs a maintainer-side correction, and fork workflow execution needs the normal approval.