All pull requests

Pull requests

• Community benchmark: Tesla P40 (Windows Server 2022 VM), four models at 256K context

  #2328·vitecopeter opened Oct 13, 2026·

  ·

• qwen35: widen F16 by hand so upload_f32 builds on CUDA 12.1

  #2327·nekomario28 opened Oct 13, 2026Contributor·

  ·
  1

• batch-groups: read a cold prompt beside the decoding slots, stall 11.9 -> 0.8 s, same work 6.4% sooner (opt-in)

  #2326·SuperDistance opened Oct 13, 2026Contributor·

  ·

• --batch: keep system-prompt roots apart from the conversations (--prompt-cache-roots, opt-in)

  #2325·brocktice opened Oct 13, 2026·

  ·

• route-resident: support 256-expert top-8 models (Qwen3.6-35B-A3B)

  #2321·atomicmilkshake opened Oct 13, 2026·

  ·

• bench: Unsloth UD-Q4_K_XL two-host RTX 5090/3090 rerun and quality checks

  #2320·petria opened Oct 13, 2026·

  ·

• SYCL verify: batch windows keep the doorbell nodes under the lean window ("parallel" crashed on a B70)

  #2319·aangelinsf opened Oct 13, 2026·

  ·

• KV cache: FP16 sliding window + attention sinks, and an optional 2-bit K / 4-bit V cache scheme (--kv q2k) [EXPERIMENTAL]

  #2318·bsorensen110 opened Oct 13, 2026Contributor·

  ·
  31

• Monitor: one line per card on a layer split (load, VRAM, temperature, power, PCIe)

  #2317·kennuardo opened Oct 13, 2026·

  ·

• fix(calibrate): keep configured vision resident during tuning

  #2312·Tsury opened Oct 13, 2026·

  ·
  1

• web: default temperature 1.0, as the model card and generation_config.json say

  #2311·kevin9327 opened Oct 13, 2026·

  ·
  1

• docker: complete deployment bundle with a management console

  #2308·jazzlee008 opened Oct 13, 2026·

  ·

• setup: write the gfx1103 exact speed switches for the opt-in install

  #2304·AlexisCarbonell opened Oct 13, 2026·

  ·
  1

• hip: a hipBLASLt tuning table for gfx1103 (Radeon 780M)

  #2303·AlexisCarbonell opened Oct 13, 2026·

  ·
  1

• hip: gfx1103 gets the native dot4 and the gfx11 WMMA kernels

  #2302·AlexisCarbonell opened Oct 13, 2026·

  ·
  1

• bench: add hosted Codex quality baseline for the quant study

  #2300·Jhacarreiro opened Oct 13, 2026·

  ·

• prefill: STRATA_GDN_CHUNKED=1 on gfx11 WMMA (opt-in; gfx1151: recurrence 1.6x, a warm 4K-128K prompt 1.4-2.1%)

  #2299·libratechw opened Oct 13, 2026Contributor·

  ·

• community benchmark: RTX 4080 SUPER, Qwen3.8-Flash-Next IQ2_XS

  #2298·johlim opened Oct 13, 2026·

  ·

• bench: run under UTF-8 and read CUDA from new nvidia-smi headers

  #2296·johlim opened Oct 13, 2026·

  ·

• web: an optional interface language (zh-CN first), English unchanged

  #2294·wulezhi-hui opened Oct 13, 2026·

  ·

• Anthropic API: cache_creation_input_tokens, the prompt tokens read now and kept for the next request

  #2290·xrip opened Oct 13, 2026·

  ·
  1

• tools: validate GGUF headers and tensor extents

  #2288·KeyanHu-git opened Oct 13, 2026·

  ·

• setup: resume downloads when HEAD omits Content-Length

  #2287·KeyanHu-git opened Oct 13, 2026·

  ·
  1

  1

• fix(vision): retain image ownership through request preparation

  #2286·Tsury opened Oct 13, 2026·

  ·

• fix(serve): preserve late instruction roles across adapters

  #2285·Tsury opened Oct 13, 2026·

  ·