Awesome Speculative Decoding

745 papers (2018–2026), 52 hand-annotated. The corpus is harvested programmatically from OpenAlex, filtered for relevance, and classified against the taxonomy in notes/taxonomy.md — so it is reproducible and re-runnable rather than a snapshot of one person's reading. This list is the companion artifact to a survey paper; the LaTeX source lives in paper/.

⭐ Start here · ⭐⭐ Must read · ⭐⭐⭐ Add-on · ◆ historical landmark

---

The one-paragraph version

Autoregressive decoding is memory-bandwidth-bound: generating one token requires streaming every weight of the model through the memory hierarchy, so the arithmetic units sit mostly idle. Scoring many tokens in one forward pass costs almost the same wall-clock time as scoring one. Speculative decoding exploits exactly this: a cheap drafter guesses the next few tokens, the expensive target model verifies them all in a single pass, and an acceptance rule commits the longest correct prefix. With the rejection-sampling rule of Leviathan et al. and Chen et al., the committed text is distributed identically to ordinary sampling from the target — the speedup is free of quality cost. Research since then has been a search along three axes: making drafts cheaper, making them more likely to be accepted, and making the whole thing survive contact with a real batched serving system.

Contents

• Reading Guide & Timeline
• Foundations & Surveys
  • Surveys & Position Papers (18)
  • Lossless Acceptance (34)
• Drafting: Where Proposals Come From
  • Independent Draft Models (120)
  • Self-Drafting (37)
  • Auxiliary Prediction Heads (52)
  • Retrieval, N-gram & Model-Free (22)
  • Multi-Token Prediction Objectives (27)
  • Cascades & Hierarchies (7)
• Verification: What Gets Committed
  • Token-Tree Verification (36)
  • Relaxed & Lossy Acceptance (12)
  • Multi-Draft & Optimal Acceptance (5)
• Training & Aligning Drafters
  • Distillation (19)
  • Online & Test-Time Adaptation (5)
  • Drafter Architecture (1)
• Systems, Serving & Hardware
  • Batching, Scheduling & Throughput (121)
  • Distributed & Pipelined Serving (13)
  • KV Cache, Offloading & Quantization (42)
  • On-Device & Edge (36)
  • Production Implementations (10)
• Applications & Modalities
  • Long Context (9)
  • Multimodal, Speech & Video (36)
  • Reasoning & Test-Time Compute (40)
  • Agents, Tool Use & Structured Output (6)
  • Retrieval-Augmented Generation (2)
  • Diffusion & Non-Autoregressive Models (12)
  • Mixture-of-Experts Targets (4)
  • Domain-Specific Applications (3)
• Theory & Evaluation
  • Speedup Models & Analysis (5)
  • Optimality & Limits (3)
  • Benchmarks & Measurement (8)
• Statistics
• Contributing

---

Reading Guide & Timeline

Follow the must-read line from left to right, then use the short picks inside each category.

⭐⭐ Must-read timeline, by first public appearance:

2018-11 Blockwise → 2022-03 SpecDec → 2022-11 Exact SD → 2023-05 SpecInfer → 2023-09 Draft & Verify → 2023-10 DistillSpec → 2023-10 Batching × SD → 2023-11 REST → 2024-01 Medusa → 2024-01 EAGLE → 2024-04 TriForce → 2024-04 MTP

---

Foundations & Surveys

Surveys & Position Papers

Prior overviews of the field.

Reading picks

• ⭐ Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding

More papers in this category

• Beyond the Speculative Game: A Survey of Speculative Execution in Large Language Models — 2024 · arXiv
• Efficient Inference for Edge Large Language Models: A Survey — 2025 · Tsinghua Science & Technology
• Collaborative Inference and Learning between Edge SLMs and Cloud LLMs: A Survey of Algorithms, Execution, and Open Challenges — 2025 · arXiv adaptive
• A Comparative Survey of Inference Acceleration for DLLMs against AR-LLMs: No Free Lunch — 2026 · Preprints.org
• A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models — 2024 · arXiv
• Accelerating Inference in Diffusion Large Language Models: A Survey — 2026 adaptive
• Adaptive Computation in the LLM Era : A Unified Survey of Routing, Cascades, and Test-Time Scaling — 2026 adaptive
• Benchmarking the Energy Savings with Speculative Decoding Strategies — 2026 · Open MIND
• Closer Look at Efficient Inference Methods: A Survey of Speculative Decoding — 2024 · arXiv
• DESIGNING HIGH-LOAD SYSTEMS WITH LARGE LANGUAGE MODELS UNDER INFERENCE — 2025 · SOFT MEASUREMENTS AND COMPUTING adaptive
• From Tensor Buffer to Distributed Memory Hierarchy: A Survey of KV Cache Management for LLM Serving — 2026 · arXiv
• I/o for LLM inference: a survey of storage and memory bottlenecks — 2026 · Artificial Intelligence Review
• Impact of Sequence Length Divergence on Speculative Decoding Efficiency with Fine-Tuned vs. Generic Drafters — 2026 · Zenodo (CERN European Organization for ...
• Minimizing Response Latency in LLM-Based Agent Systems: A Comprehensive Survey — 2026 · IEEE Access adaptive
• Mitigating Disparate Speed-Up Rates in Speculative Decoding for Multitask Models via Drafter Fine-Tuning — 2026 · Zenodo (CERN European Organization for ...
• Semantic Similarity of Fine-Tuning Data and Speculative Decoding Degradation in Qwen2.5 Multitask Instruction-Following — 2026 · Zenodo (CERN European Organization for ...
• Speculative Decoding and Beyond: An In-Depth Survey of Techniques — 2025 · arXiv

Lossless Acceptance

The rejection-sampling rule that makes speculation exact, and its analyses.

Reading picks

• ⭐⭐ Fast Inference from Transformers via Speculative Decoding
• ⭐⭐⭐ Accelerating Large Language Model Decoding with Speculative Sampling

More papers in this category

• Speculative Decoding via Early-exiting for Faster LLM Inference with Thompson Sampling Control Mechanism — 2024 · arXiv lossless
• BiTA: Bi-directional tuning for lossless acceleration in large language models — 2024 · Expert Systems with Applications lossless tree
• Efficient Inference for Large Language Model-based Generative Recommendation — 2024 · arXiv lossless
• Chimera: A Lossless Decoding Method for Accelerating Large Language Models Inference by Fusing all Tokens — 2024 · arXiv lossless
• Generation Meets Verification: Accelerating Large Language Model Inference with Smart Parallel Auto-Correct Decoding — 2024 · arXiv lossless
• Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding — 2024 · arXiv lossless training-free adaptive
• Scaling LLM Speculative Decoding: Non-Autoregressive Forecasting in Large-Batch Scenarios — 2025 · Proceedings of the AAAI Conference on A... lossless tree
• A Practical Investigation of Training-free Relaxed Speculative Decoding — 2026 · arXiv lossless training-free
• Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies — 2025 · arXiv lossless
• AdaEAGLE: Optimizing Speculative Decoding via Explicit Modeling of Adaptive Draft Structures — 2024 · arXiv lossless adaptive
• Amortized Computation: A Theory of Inference, Search, and Policy Improvement — 2026 · Zenodo (CERN European Organization for ... lossless adaptive
• Block Verification Accelerates Speculative Decoding — 2024 · arXiv lossless
• Boosting Lossless Speculative Decoding via Feature Sampling and Partial Alignment Distillation — 2024 · arXiv lossless
• Distributed Generative Inference of LLM at Internet Scales with Multi-Dimensional Communication Optimization — 2026 · arXiv lossless adaptive
• Distributed Speculative Inference (DSI): Speculation Parallelism for Provably Faster Lossless Language Model Inference — 2024 · arXiv lossless
• EasySpec: Layer-Parallel Speculative Decoding for Efficient Multi-GPU Utilization — 2025 · arXiv lossless training-free
• FREE: Uncertainty-Aware Autoregression for Parallel Diffusion Transformers — 2025 · arXiv lossless
• From Quarter to All: Accelerating Speculative LLM Decoding via Floating-Point Exponent Remapping and Parameter Sharing — 2025 · arXiv lossless
• HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding — 2026 · Open MIND lossless training-free
• Lossless Acceleration of Large Language Models with Hierarchical Drafting based on Temporal Locality in Speculative Decoding — 2025 · arXiv lossless
• Lossless but Not Free: An Empirical Anatomy of Speculative Decoding on Consumer Hardware — 2026 · arXiv lossless
• Multi-Token Residual Prediction — 2026 · arXiv lossless adaptive
• Overcoming Joint Intractability with Lossless Hierarchical Speculative Decoding — 2026 · arXiv lossless
• ParallelVLM: Lossless Video-LLM Acceleration with Visual Alignment Aware Parallel Speculative Decoding — 2026 · arXiv lossless training-free
• READER: Retrieval-Assisted Drafter for Efficient LLM Inference — 2025 · arXiv lossless
• Sparse-to-Dense: A Free Lunch for Lossless Acceleration of Video Understanding in LLMs — 2025 · arXiv lossless training-free
• TAPS: Target-Aware Prefix Tree Selection for Diffusion-Drafted Speculative Decoding — 2026 · arXiv lossless tree
• TS-DP: Reinforcement Speculative Decoding For Temporal Adaptive Diffusion Policy Acceleration — 2025 · arXiv lossless adaptive
• TokenSwift: Lossless Acceleration of Ultra Long Sequence Generation — 2025 · arXiv lossless adaptive
• Towards Better & Faster Autoregressive Image Generation: From the Perspective of Entropy — 2025 · arXiv lossless adaptive
• UniSpec: Training-Free Speculative Decoding for Robust LLM Acceleration Across Languages and Hardware — 2026 · Underline Science Inc. lossless training-free
• UniVer: A Unified Perspective for Multi-step and Multi-draft Speculative Decoding — 2026 · arXiv lossless tree adaptive

Drafting: Where Proposals Come From

Independent Draft Models

A separate smaller model proposes; the classic two-model setup.

Reading picks

• ⭐⭐ Speculative Decoding: Exploiting Speculative Execution for Accelerating Seq2seq Generation

More papers in this category

• Distributed Adaptive Speculative Decoding: Accelerating Large Language Model Inference With Context-Aware Draft Selection — 2026 adaptive
• SW-SpeedDLM: Sliding Window Speculative Decoding for Diffusion Language Models Under Long Context Constraints — 2026 · Mathematics
• FlexSpec: Frozen Drafts Meet Evolving Targets in Edge-Cloud Collaborative LLM Speculative Decoding — 2026 · IEEE Transactions on Mobile Computing adaptive
• RASD: Retrieval-Augmented Speculative Decoding — 2025 · arXiv
• Dovetail: A CPU/GPU Heterogeneous Speculative Decoding for LLM inference — 2024 · Open MIND lossless adaptive
• Hierarchical Speculative Decoding with Dynamic Window — 2025 adaptive
• SpecDec++: Boosting Speculative Decoding via Adaptive Candidate Lengths — 2024 · arXiv adaptive
• Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters — 2024 · arXiv
• Accelerating Autoregressive Speech Synthesis Inference With Speech Speculative Decoding — 2025 · arXiv
• Draft Model Knows When to Stop: Self-Verification Speculative Decoding for Long-Form Generation — 2024 · Open MIND training-free adaptive
• SpecVLM: Enhancing Speculative Decoding of Video LLMs via Verifier-Guided Token Pruning — 2025 · Open MIND training-free
• SpecBranch: Speculative Decoding via Hybrid Drafting and Rollback-Aware Branch Parallelism — 2025 · arXiv adaptive
• Speculative Verification: Exploiting Information Gain for Speculative Decoding — 2026 · Underline Science Inc.
• 3-Model Speculative Decoding — 2025 · arXiv
• A 28nm Speculative-Decoding LLM Processor Achieving 105-to-685µs/Token Latency for Billion-Parameter Models — 2026
• A Drop-In Solution for On-the-Fly Adaptation of Speculative Decoding in Large Language Models — 2025 adaptive
• AASD: Accelerate Inference by Aligning Speculative Decoding in Multimodal Large Language Models — 2025
• Accelerated Diffusion Models via Speculative Sampling — 2025 · arXiv
• Accelerated Test-Time Scaling with Model-Free Speculative Sampling — 2025 · Open MIND training-free adaptive
• Accelerating Greedy Coordinate Gradient and General Prompt Optimization via Probe Sampling — 2024 · arXiv
• Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding — 2026 · arXiv lossless
• Accelerating Speculative Decoding with Block Diffusion Draft Trees — 2026 · arXiv tree
• Accelerating inference in genomic and proteomic foundation models via speculative decoding — 2026 · bioRxiv (Cold Spring Harbor Laboratory)
• Acceptance-Guided Adaptive Speculative Decoding for Efficient Large Language Model Inference — 2026 adaptive
• Accepted Prefixes Are Not All You Need: A Negative Result on PEFT-Based Block-Diffusion Drafting — 2026 · arXiv
• AdaEDL: Early Draft Stopping for Speculative Decoding of Large Language Models via an Entropy-based Lower Bound on Token Acceptance Probability — 2024 · arXiv training-free adaptive
• AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters — 2026 · arXiv adaptive
• AdaSD: Adaptive Speculative Decoding for Efficient Language Model Inference — 2025 · arXiv adaptive
• AdaSPEC: Selective Knowledge Distillation for Efficient Speculative Decoders — 2025 · arXiv
• Arbitrage: Efficient Reasoning via Advantage-Aware Speculation — 2025 · arXiv
• Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding — 2026 · Open MIND
• Beyond the Target: From Imitation to Collaboration in Speculative Decoding — 2026 · arXiv
• BlockPilot: Instance-Adaptive Policy Learning for Diffusion-based Speculative Decoding — 2026 · arXiv lossless training-free adaptive
• CARD: A cache-assisted parallel speculative decoding framework via query-and-correct paradigm for accelerating LLM inference — 2026 · High-Confidence Computing
• ConFu: Contemplate the Future for Better Speculative Sampling — 2026 · arXiv adaptive
• ConfSpec: Efficient Step-Level Speculative Reasoning via Confidence-Gated Verification — 2026 · Open MIND
• Context-Aware Assistant Selection for Improved Inference Acceleration with Large Language Models — 2024 · arXiv
• Cross-Family Speculative Decoding for Polish Language Models on AppleSilicon: An Empirical Evaluation of Bielik11B with UAG-Extended MLX-LM — 2026 · arXiv
• Cross-Family Speculative Prefill: Training-Free Long-Context Compression with Small Draft Models — 2026 · arXiv training-free
• DART: Diffusion-Inspired Speculative Decoding for Fast LLM Inference — 2026 · Open MIND lossless
• … and 79 more in data/papers.yaml under primary: draft-independent.

Self-Drafting

The target model drafts for itself via early exit, layer skipping, or Jacobi iteration.

Reading picks

• ⭐⭐ Draft& Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding
• ⭐⭐⭐ Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
• ⭐⭐⭐ LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding

More papers in this category

• CLLMs: Consistency Large Language Models — 2024 · arXiv
• Kangaroo: Lossless Self-Speculative Decoding via Double Early Exiting — 2024 · arXiv lossless
• SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration — 2024 · arXiv training-free adaptive
• KNN-SSD: Enabling Dynamic Self-Speculative Decoding via Nearest Neighbor Layer Set Optimization — 2025 · Underline Science Inc. adaptive
• S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs — 2024 · arXiv
• CLaSp: In-Context Layer Skip for Self-Speculative Decoding — 2025 · arXiv training-free adaptive
• Draft on the Fly: Adaptive Self-Speculative Decoding using Cosine Similarity — 2024 · arXiv training-free adaptive
• Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting — 2024 lossless
• Accelerating Large Language Model Inference with Self-Supervised Early Exits — 2024 · arXiv adaptive
• Accelerating Speculative Diffusions via Block Verification — 2026 · arXiv training-free
• Cassandra: Enabling Reasoning LLMs at Edge via Self-Speculative Decoding — 2026 · arXiv lossless training-free
• Component-Aware Self-Speculative Decoding in Hybrid Language Models — 2026 · arXiv
• ConfLayers: Adaptive Confidence-based Layer Skipping for Self-Speculative Decoding — 2026 · arXiv training-free adaptive
• DEL: Context-Aware Dynamic Exit Layer for Efficient Self-Speculative Decoding — 2025 · arXiv training-free adaptive
• Experience-Driven Dynamic Exits for LLMs with Reinforcement Learning — 2026 · arXiv adaptive
• Fast SceneScript: Fast and Accurate Language-Based 3D Scene Understanding via Multi-Token Prediction — 2025 · arXiv
• FastVLM: Self-Speculative Decoding for Fast Vision-Language Model Inference — 2025 · Open MIND
• Faster Language Models with Better Multi-Token Prediction Using Tensor Decomposition — 2024 · arXiv
• FractalLLM: Lossless Self-Speculative Decoding with Layer Embedded Self-Compression — 2025 lossless
• Gearbx: Entropy-Routed Dynamic Quantization for LLM Inference — 2026 · Zenodo (CERN European Organization for ... adaptive
• M2R2: Mixture of Multi-Rate Residuals for Efficient Transformer Inference — 2025 · arXiv
• MTEDS: Memory and Time Efficient Speculative Decoding with Dynamic Sparsity and Bypass Scheduling — 2026 adaptive
• Next-Latent Prediction Transformers Learn Compact World Models — 2025 · arXiv
• Pre-Training Curriculum for Multi-Token Prediction in Language Models — 2025 · arXiv
• Quasar: Quantized Self-Speculative Acceleration for Rapid Inference via Memory-Efficient Verification — 2026 · arXiv training-free
• RAD: Redundancy-Aware Distillation for Hybrid Models via Self-Speculative Decoding — 2025 · arXiv
• S2D2: Fast Decoding for Diffusion LLMs via Training-Free Self-Speculation — 2026 · arXiv training-free adaptive
• SNLP: Layer-Parallel Inference via Structured Newton Corrections — 2026 · arXiv
• SPORK: Self-Speculative Forking to Accelerate Agentic LLM Inference — 2026 · arXiv training-free
• Self Speculative Decoding for Diffusion Large Language Models — 2025 · arXiv lossless
• Self-Speculative Decoding for On-device MoE Acceleration — 2026 lossless adaptive
• Self-Speculative Masked Diffusions — 2025 · arXiv
• SpecBound: Adaptive Bounded Self-Speculation with Layer-wise Confidence Calibration — 2026 · arXiv adaptive
• Unlocking the Edge deployment and ondevice acceleration of multi-LoRA enabled one-for-all foundational LLM — 2026 · Underline Science Inc. tree adaptive

Auxiliary Prediction Heads

Lightweight heads on the target's hidden states (Medusa, EAGLE, and descendants).

Reading picks

• ⭐⭐ Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
• ⭐⭐ EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
• ⭐⭐⭐ EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test

More papers in this category

• Falcon: Faster and Parallel Inference of Large Language Models Through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree — 2024 · Proceedings of the AAAI Conference on A... lossless
• Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding — 2024 · arXiv
• Recurrent Drafter for Fast Speculative Decoding in Large Language Models — 2024 · arXiv tree adaptive
• ResDecode: Accelerating Large Language Models Inference via Residual Decoding Heads — 2025 · Big Data Mining and Analytics
• AdaSpec: Adaptive Multilingual Speculative Decoding with Self-Synthesized Language-Aware Training and Vocabulary Simplification — 2026 · Proceedings of the AAAI Conference on A... adaptive
• Dynamic Depth Decoding: Faster Speculative Decoding for LLMs — 2024 · arXiv lossless tree adaptive
• A Markov Categorical Framework for Language Modeling — 2025 · arXiv
• Acceleration Multiple Heads Decoding for LLM via Dynamic Tree Attention — 2025 · arXiv tree adaptive
• Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference — 2024 · arXiv
• Boosting the Performance of Tree-Based Speculative Decoding of LLMs on FPGAs — 2026 tree
• C2T: A Classifier-Based Tree Construction Method in Speculative Decoding — 2025 · arXiv adaptive
• CORAL: Learning Consistent Representations across Multi-step Training with Lighter Speculative Drafter — 2025 · arXiv
• Cerberus: Efficient Inference with Adaptive Parallel Decoding and Sequential Knowledge Enhancement — 2024 · arXiv adaptive
• Clover: Regressive Lightweight Speculative Decoding with Sequential Knowledge — 2024 · arXiv
• ComplianceNLP: Knowledge-Graph-Augmented RAG for Multi-Framework Regulatory Gap Detection — 2026 · arXiv
• Cross-Attention Speculative Decoding — 2025 · arXiv
• Draft Model Knows When to Stop: A Self-Verification Length Policy for Speculative Decoding — 2024 · Qeios training-free adaptive
• EAGLIZE: Improving Stability of Knowledge Distillation — 2026 adaptive
• Efficient Speculative Decoding for Llama at Scale: Challenges and Solutions — 2025 · arXiv tree
• EvoSpec: Evolving Speculative Decoding via Real-Time Vocabulary and Parameter Adaptation — 2026 · arXiv adaptive
• FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling — 2025 · arXiv
• Fail Fast, Win Big: Rethinking the Drafting Strategy in Speculative Decoding via Diffusion LLMs — 2025 · Open MIND lossless
• FastEagle: Cascaded Drafting for Accelerating Speculative Decoding — 2026 · arXiv lossless tree
• Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding — 2025 · arXiv
• HeteroSpec: Leveraging Contextual Heterogeneity for Efficient Speculative Decoding — 2026 · Underline Science Inc. adaptive
• KOALA: Enhancing Speculative Decoding for LLM via Multi-Layer Draft Heads with Adversarial Learning — 2024 · arXiv
• KVSpec: Efficient KV-Cache Reuse for Speculative Decoding — 2025
• Learning Harmonized Representations for Speculative Sampling — 2024 · arXiv
• MARS: Enabling Autoregressive Models Multi-Token Generation — 2026 · arXiv
• Mixture of Attentions For Speculative Decoding — 2024 · arXiv
• MoE-Spec: Expert Budgeting for Efficient Speculative Decoding — 2026 · Open MIND training-free
• NanoSpec: Accelerating Speculative Decoding using Minimalist In-Context Vocabularies — 2026 · arXiv training-free
• Note on Martingale Theory and Applications — 2026 · arXiv adaptive
• P-EAGLE: Parallel-Drafting EAGLE with Scalable Training — 2026 · Open MIND
• PARD: Accelerating LLM Inference with Low-Cost PARallel Draft Model Adaptation — 2025 · arXiv
• PLD+: Accelerating LLM Inference by Leveraging Language Model Artifacts — 2024 · arXiv
• Pair-In, Pair-Out: Latent Multi-Token Prediction for Efficient LLMs — 2026 · arXiv
• SlimSpec: Low-Rank Draft LM-Head for Accelerated Speculative Decoding — 2026 · arXiv adaptive
• SpecBlock: Block-Iterative Speculative Decoding with Dynamic Tree Drafting — 2026 · arXiv tree adaptive
• SpecLA: Efficient Speculative Decoding for Linear-Attention Models — 2026 · arXiv
• … and 9 more in data/papers.yaml under primary: draft-heads.

Retrieval, N-gram & Model-Free

Proposals copied from the prompt or a datastore; no draft forward pass at all.

Reading picks

• ⭐⭐ REST: Retrieval-Based Speculative Decoding
• ⭐⭐⭐ Inference with Reference: Lossless Acceleration of Large Language Models

More papers in this category

• Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding — 2024 · arXiv training-free
• SAM Decoding: Speculative Decoding via Suffix Automaton — 2024 · arXiv lossless adaptive
• Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling — 2024 · arXiv lossless training-free tree
• Efficiency Unleashed: Inference Acceleration for LLM-based Recommender Systems with Speculative Decoding — 2024 · arXiv lossless
• A-IO: Adaptive Inference Orchestration for Memory-Bound NPUs — 2026 · arXiv adaptive
• AdaPLD: Adaptive Retrieval and Reuse for Efficient Model-Free Speculative Decoding — 2026 · arXiv training-free adaptive
• Breaking Codebook Redundancy for Faster Autoregressive Image Generation with Retrieval-Augmented Speculative Decoding — 2026 training-free
• CREST: Effectively Compacting a Datastore For Retrieval-Based Speculative Decoding — 2024 · arXiv
• DReSD: Dense Retrieval for Speculative Decoding — 2025
• Draft Less, Retrieve More: Hybrid Tree Construction for Speculative Decoding — 2026 · arXiv lossless training-free adaptive
• Faster In-Context Learning for LLMs via N-Gram Trie Speculative Decoding — 2025 · Open MIND
• Goose: Anisotropic Speculation Trees for Training-Free Speculative Decoding — 2026 · arXiv lossless training-free adaptive
• LogitSpec: Accelerating Retrieval-based Speculative Decoding via Next Next Token Speculation — 2026 · arXiv training-free
• RACER: Retrieval-Augmented Contextual Rapid Speculative Decoding — 2026 · Underline Science Inc. training-free
• Retrieval-Based Speculative Decoding For Autoregressive Speech Synthesis — 2026 training-free tree
• SENSE: Semantic Embedding Navigation with Soft-gated Evaluation for Retrieval-based Speculative Decoding — 2026 · arXiv training-free
• SSSD: Simply-Scalable Speculative Decoding — 2026 · Underline Science Inc. training-free
• Task-Centric Acceleration of Small-Language Models — 2026 · Open MIND training-free adaptive
• ToolSpec: Accelerating Tool Calling via Schema-Aware and Retrieval-Augmented Speculative Decoding — 2026 · arXiv training-free
• Well Begun is Half Done: Low-resource Preference Alignment by Weak-to-Strong Decoding — 2025 · arXiv

Multi-Token Prediction Objectives

Pretraining the model to emit several tokens per step.

Reading picks

• ⭐⭐ Blockwise Parallel Decoding for Deep Autoregressive Models
• ⭐⭐ Better & Faster Large Language Models via Multi-token Prediction
• ⭐⭐⭐ DeepSeek-V3 Technical Report

More papers in this category

• Accelerating Codec-based Speech Synthesis with Multi-Token Prediction and Speculative Decoding — 2024 · arXiv
• MiMo-V2-Flash Technical Report — 2026 · arXiv
• AI Model Spotlight: ** NVIDIA Nemotron 3 Super — 2026 · Open MIND adaptive
• Attention Drift: What Autoregressive Speculative Decoding Models Learn — 2026 · arXiv
• Breaking Entropy Bounds: Accelerating RL Training via MTP with Rejection Sampling — 2026 · arXiv
• DynaMo: Accelerating Language Model Inference with Dynamic Multi-Token Sampling — 2024 · arXiv adaptive
• Efficient Training-Free Multi-Token Prediction via Embedding-Space Probing — 2026 · arXiv lossless training-free tree adaptive
• Exploring and Improving Drafts in Blockwise Parallel Decoding — 2024 · arXiv
• Fast and Expressive Multi-Byte Prediction with Probabilistic Circuits — 2025 · arXiv
• FastMTP: Accelerating LLM Inference with Enhanced Multi-Token Prediction — 2025 · arXiv lossless adaptive
• HPD-Parsing: Hierarchical Parallel Document Parsing — 2026 · arXiv
• Llasa+: Free Lunch for Accelerated and Streaming Llama-Based Speech Synthesis — 2025 · arXiv training-free
• MTP-RL: Acceleration of Reinforcement Learning Rollouts with Policy-Aligned Multi-Token Prediction — 2026 · Underline Science Inc.
• Multi-Token Prediction via Self-Distillation — 2026 · Open MIND
• Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning — 2026 · arXiv
• Nemotron-Labs-Diffusion: A Tri-Mode Language Model Unifying Autoregressive, Diffusion, and Self-Speculation Decoding — 2026 · arXiv
• P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling — 2026 · arXiv adaptive
• Reinforcement Speculative Decoding for Fast Ranking — 2025 · DR-NTU (Nanyang Technological University)
• SJD++: Improved Speculative Jacobi Decoding for Training-free Acceleration of Discrete Auto-regressive Text-to-Image Generation — 2025 · IEEE Transactions on Pattern Analysis a... training-free
• Self-Distillation for Multi-Token Prediction — 2026 · arXiv
• Speculative Pipeline Decoding: Higher-Accruacy and Zero-Bubble Speculation via Pipeline Parallelism — 2026 · arXiv
• TextSeal: A Localized LLM Watermark for Provenance & Distillation Protection — 2026 · arXiv
• Windowed-MTP: Removing the Full-Context Draft-KV Tax at Million-Token Context — 2026 · arXiv lossless training-free
• Your LLM Knows the Future: Uncovering Its Multi-Token Prediction Potential — 2025 · arXiv lossless

Cascades & Hierarchies

Three or more tiers; drafters that themselves speculate.

Reading picks

• ⭐ Accelerating LLM Inference with Staged Speculative Decoding

More papers in this category

• Cascade Speculative Drafting for Even Faster LLM Inference — 2023 · arXiv
• CASCADE: Context-Aware Relaxation for Speculative Image Decoding — 2026 · arXiv tree
• D^2SD: Accelerating Speculative Decoding with Dual Diffusion Draft Models — 2026 · arXiv
• Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models — 2026 · arXiv
• RLM-Cascade: Response-Level Speculative Decoding for Cost-Efficient LLM API Serving — 2026 · arXiv
• SENTRY-DVL (v1.1.1): A Cascaded Hybrid Guardrail Architecture for Fact-Verification and Regulatory Compliance under the EU AI Act — 2026 · Zenodo (CERN European Organization for ...

Verification: What Gets Committed

Token-Tree Verification

Verify many branches at once under a tree attention mask.

Reading picks

• ⭐⭐ SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
• ⭐⭐⭐ Sequoia: Scalable, Robust, and Hardware-aware Speculative Decoding
• ⭐⭐⭐ EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees

More papers in this category

• SpecInfer: Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification — 2024 tree
• OPT-Tree: Speculative Decoding with Adaptive Draft Tree Structure — 2024 · Transactions of the Association for Com... lossless tree adaptive
• Dynamic-Width Speculative Beam Decoding for LLM Inference — 2025 · Proceedings of the AAAI Conference on A... tree adaptive
• ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding — 2024 · arXiv tree adaptive
• DySpec: Faster speculative decoding with dynamic token tree structure — 2024 · World Wide Web tree adaptive
• Improving Multi-candidate Speculative Decoding — 2024 · arXiv tree adaptive
• SpecMamba: Accelerating Mamba Inference on FPGA with Speculative Decoding — 2025 tree
• ASDT: Adaptive Speculative Decoding Tree Based on Token Acceptance Feedback — 2025 training-free tree adaptive
• Bastion: Budget-Aware Speculative Decoding with Tree-structured Block Diffusion Drafting — 2026 · arXiv training-free tree adaptive
• Cost-Aware Diffusion Draft Trees for Speculative Decoding — 2026 · arXiv tree
• DIAA: A Decoding-Efficient Inference Acceleration Approach for On-Device Large Language Models — 2026 · Proceedings of the AAAI Conference on A... training-free tree adaptive
• DeFT: Decoding with Flash Tree-attention for Efficient Tree-structured LLM Inference — 2024 · arXiv tree
• Dynamic-Width Speculative Beam Decoding for Efficient LLM Inference — 2024 · arXiv tree adaptive
• EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction — 2026 · arXiv training-free tree
• Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoE — 2025 · arXiv tree
• JetFlow: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting — 2026 · arXiv tree
• JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting — 2026 · arXiv tree
• Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training — 2024 · arXiv tree
• Making Every Verified Token Count: Adaptive Verification for MoE Speculative Decoding — 2026 · arXiv lossless training-free tree adaptive
• ONE-PASS: Single Forward Pass Decoding for Listwise Reranking — 2026 tree
• Parallel Decoding via Hidden Transfer for Lossless Large Language Model Acceleration — 2024 · arXiv lossless tree
• Re-SpS: A Reinforcement Learning Approach to Speculative Sampling — 2026 · Proceedings of the AAAI Conference on A... tree
• Recursive Speculative Decoding: Accelerating LLM Inference via Sampling Without Replacement — 2024 · arXiv tree
• SAGE: Accelerating Vision-Language Models via Entropy-Guided Adaptive Speculative Decoding — 2026 · Open MIND lossless adaptive
• STree: Speculative Tree Decoding for Hybrid State-Space Models — 2025 · arXiv tree
• SpecASR: Accelerating LLM-based Automatic Speech Recognition via Speculative Decoding — 2025 lossless tree adaptive
• Speculative Sampling with Reinforcement Learning — 2026 · arXiv tree
• SuperSpec: Enhanced Verification and Sampling for End-to-End LLM Speculative Decoding — 2025 · arXiv tree adaptive
• TALON: Confidence-Aware Speculative Decoding with Adaptive Token Trees — 2026 · arXiv training-free tree adaptive
• Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters — 2024 · arXiv tree
• TreeFlash: Parallel AR-Approximation for Faster Speculative Decoding — 2026 · arXiv tree
• Unlocking Parallelism in Autoregressive Language Models via Speculative Decoding with Progressive Tree Drafting — 2026 · arXiv training-free
• Yggdrasil: Bridging Dynamic Speculation and Static Runtime for Latency-Optimal Tree-Based LLM Decoding — 2025 · arXiv tree adaptive

Relaxed & Lossy Acceptance

Trading exact distribution preservation for more accepted tokens.

Reading picks

• ⭐ Speculative Decoding with Big Little Decoder

More papers in this category

• Fast and High-Quality Auto-Regressive Speech Synthesis via Speculative Decoding — 2024 · arXiv
• LANTERN++: Enhancing Relaxed Speculative Decoding with Static Tree Drafting for Visual Auto-regressive Models — 2025 · arXiv adaptive
• AutoJudge: Judge Decoding Without Manual Annotation — 2025 · arXiv
• Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling — 2026 · arXiv
• Efficient Adaptive Rejection Sampling for Accelerating Speculative Decoding in Large Language Models — 2025 · arXiv adaptive
• HeiSD: Hybrid Speculative Decoding for Embodied Vision-Language-Action Models with Kinematic Awareness — 2026 · arXiv
• LANTERN: Accelerating Visual Autoregressive Models with Relaxed Speculative Decoding — 2024 · arXiv
• List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compression — 2025 · arXiv
• Self-Speculative Decoding for LLM-based ASR with CTC Encoder Drafts — 2026 · arXiv
• Spec-VLA: Speculative Decoding for Vision-Language-Action Models with Relaxed Acceptance — 2025 · Open MIND
• When Is a Draft Accepted? A Theory of Acceptance in Speculative Decoding — 2026 · arXiv lossless tree

Multi-Draft & Optimal Acceptance

Several proposals per step; the optimal rule is an optimal-transport problem.

Reading picks

• ⭐ SpecTr: Fast Speculative Decoding via Optimal Transport

More papers in this category

• Multi-candidate Speculative Decoding — 2024 · Lecture notes in computer science
• Action Deviation-Aware Inference for Low-Latency Wireless Robots — 2025 · arXiv
• SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding — 2024 · arXiv
• TABED: Test-Time Adaptive Ensemble Drafting for Robust Speculative Decoding in LVLMs — 2026 · Open MIND training-free adaptive

Training & Aligning Drafters

Distillation

Acceptance rate is an alignment metric, so distil the drafter toward the target.

Reading picks

• ⭐⭐ DistillSpec: Improving Speculative Decoding via Knowledge Distillation
• ⭐⭐⭐ FastDraft: How to Train Your Draft

More papers in this category

• The Mamba in the Llama: Distilling and Accelerating Hybrid Models — 2024 · arXiv
• LoRAExit: Empowering Dynamic Modulation of LLMs in Resource-limited Settings using Low-rank Adapters — 2024 adaptive
• Seedream 4.0: Toward Next-generation Multimodal Image Generation — 2025 · arXiv
• 48 - Steering Pretrained Drafters During Speculative Decoding — 2026 · Open MIND adaptive
• Clover-2: Accurate Inference for Regressive Lightweight Speculative Decoding — 2024 · arXiv
• DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation — 2026 · Underline Science Inc. adaptive
• Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs — 2024 · arXiv
• Drift-AR: Single-Step Visual Autoregressive Generation via Anti-Symmetric Drifting — 2026 · arXiv
• Knowledge Distillation with Training Wheels — 2025 · Communications in computer and informat...
• Playing with Transformer at 30+ FPS via Next-Frame Diffusion — 2025 · arXiv
• Provenance Generalization and Verification Scaling for Neural Network Forensics — 2026 · Zenodo (CERN European Organization for ... adaptive
• Quantize the Target, Quantize the Drafter: Efficient Inference with Qwen3.5-4B — 2026 · arXiv
• SOAEsV2-7B/72B: Full-Pipeline Optimization for State-Owned Enterprise LLMs via Continual Pre-Training, Domain-Progressive SFT and Distillation-Enhanced Speculative Decoding — 2025 · Communications in computer and informat...
• Self-Data Distillation for Recovering Quality in Pruned Large Language Models — 2024 · arXiv
• Speculative Decoding Across Languages — 2026 · arXiv
• Steering Pretrained Drafters during Speculative Decoding — 2025 · Underline Science Inc. adaptive
• TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding — 2026 · arXiv
• Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit — 2025 · arXiv training-free

Online & Test-Time Adaptation

Adapt the drafter to the live query distribution.

Reading picks

• ⭐ Online Speculative Decoding

More papers in this category

• BanditSpec: Adaptive Speculative Decoding via Bandit Algorithms — 2025 · arXiv training-free adaptive
• CAS-Spec: Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs — 2025 · arXiv lossless tree adaptive
• Draft, Verify, and Improve: Toward Training-Aware Speculative Decoding — 2025 · arXiv lossless
• SpecFuse: Ensembling Large Language Models via Next-Segment Prediction — 2024 · arXiv training-free adaptive

Drafter Architecture

Capacity, latency, and design-space choices.

Reading picks

• ⭐ EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test

More papers in this category

• Adaptive Tree-of-Thoughts with Eagle3-Accelerated GPT-120B: A Resilient Inference Architecture for Competition Mathematical Reasoning — 2026 · Zenodo (CERN European Organization for ... adaptive

Systems, Serving & Hardware

Batching, Scheduling & Throughput

Speculation spends FLOPs to buy latency; the trade inverts at large batch.

Reading picks

• ⭐⭐ The Synergy of Speculative Decoding and Batching in Serving Large Language Models
• ⭐⭐⭐ Decoding Speculative Decoding
• ⭐⭐⭐ TurboSpec: Closed-loop Speculation Control System for Optimizing LLM Serving Goodput

More papers in this category

• PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System — 2025 adaptive
• PipeInfer: Accelerating LLM Inference using Asynchronous Pipelined Speculation — 2024 · arXiv
• SLED: A Speculative LLM Decoding Framework for Efficient Edge Serving — 2025 · arXiv
• SpecEE: Accelerating Large Language Model Inference with Speculative Early Exiting — 2025 · arXiv
• SkipDecode: Autoregressive Skip Decoding with Batching and Caching for Efficient LLM Inference — 2023 · arXiv
• MoE-Inference-Bench: Performance Evaluation of Mixture of Expert Large Language and Vision Models — 2025
• Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inference — 2024 · Open MIND adaptive
• PipeSpec: Breaking Stage Dependencies in Hierarchical LLM Decoding — 2025 · arXiv
• 3D-TokSIM: Stacking 3D Memory with Token-Stationary Compute-in-Memory for Speculative LLM Inference — 2025
• BASS: Batched Attention-optimized Speculative Sampling — 2024 · arXiv
• Quantize-Sample-and-Verify: LLM Acceleration via Adaptive Edge-Cloud Speculative Decoding — 2025 · IEEE Communications Letters adaptive
• Towards Efficient LLM Inference via Collective and Adaptive Speculative Decoding — 2025 adaptive
• A Practical Tensor-Network Compression Pipeline for Production-Scale Large Language Models — 2026 · Open MIND
• A cognitive synergetic hierarchical framework for UAV swarm combat via speculative inference and role-decoupled reinforcement learning — 2026 · Frontiers in Neurorobotics
• APAR: LLMs Can Do Auto-Parallel Auto-Regressive Decoding — 2024 · arXiv
• Accelerated AI Inference via Dynamic Execution Methods — 2024 · arXiv adaptive
• Accelerating Diffusion LLMs via Adaptive Parallel Decoding — 2025 · arXiv adaptive
• Accelerating Large Language Model Inference via Early-Exiting Algorithms — 2025 · arXiv adaptive
• Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding — 2025 · arXiv adaptive
• AdaDecode: Accelerating LLM Decoding with Adaptive Layer Parallelism — 2025 · arXiv adaptive
• AdaServe: Accelerating Multi-SLO LLM Serving with SLO-Customized Speculative Decoding — 2025 · arXiv
• Adaptive Draft Sequence Length: Enhancing Speculative Decoding Throughput on PIM-Enabled Systems — 2026 adaptive
• Adaptive Repetition Controller (ARC): Decode-Time Behavioral Probes for Controllable and Efficient LLM Inference — 2026 · Zenodo (CERN European Organization for ... adaptive
• Adaptive hybrid speculative decoding for accelerating large language model inference — 2026 · Neurocomputing adaptive
• An Interpretable Latency Model for Speculative Decoding in LLM Serving — 2026 · arXiv
• Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI — 2025 · arXiv adaptive
• BitLM: Unlocking Multi-Token Language Generation with Bitwise Continuous Diffusion — 2026 · arXiv
• BubbleSpec: Turning Long-Tail Bubbles into Speculative Rollout Drafts for Synchronous Reinforcement Learning — 2026 · arXiv
• Budget-Obedient Hysteretic Gating for LLM Inference — 2026 · Zenodo (CERN European Organization for ... adaptive
• Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference — 2026 · Underline Science Inc. training-free
• CoVSpec: Efficient Device-Edge Co-Inference for Vision-Language Models via Speculative Decoding — 2026 · arXiv training-free adaptive
• Collaborative Speculative Inference for Efficient LLM Inference Serving — 2025 · arXiv adaptive
• ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Serving — 2026 · arXiv
• Continuous Discovery of Vulnerabilities in LLM Serving Systems with Fuzzing — 2026 · arXiv
• Counting What You Have Not Yet Said — 2026 · Zenodo (CERN European Organization for ... adaptive
• DSD: A Distributed Speculative Decoding Solution for Edge-Cloud Agile Large Model Serving — 2025 · arXiv adaptive
• DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation — 2026 · arXiv adaptive
• DeltaCert: Calibrated Divergence Certification for LLM Serving Systems — 2026 · Zenodo (CERN European Organization for ...
• DiP-SD: Distributed Pipelined Speculative Decoding for Efficient LLM Inference at the Edge — 2026 · arXiv
• Distilling Large Language Models for Network Active Queue Management — 2025 · arXiv adaptive
• … and 78 more in data/papers.yaml under primary: sys-batching.

Distributed & Pipelined Serving

Overlapping draft and verify across devices.

Reading picks

• ⭐ PEARL: Parallel Speculative Decoding with Adaptive Draft Length

More papers in this category

• Demystifying AI Platform Design for Distributed Inference of Next-Generation LLM models — 2024 · arXiv
• Compression with distributional and sequential constraints — 2025 · Open MIND
• DSSD: Efficient Edge-Device LLM Deployment and Collaborative Inference via Distributed Split Speculative Decoding — 2025 · arXiv
• EdgeSpec: Distributed Speculative Decoding for Large Language Models at Edge — 2026
• GAPI: Graph-Aware Prefill Interleaving for Knowledge Graph-Augmented LLM Inference in Explainable Education Policy Analysis — 2026
• Jupiter: Fast and Resource-Efficient Collaborative Inference of Generative LLMs on Edge Devices — 2025 · arXiv
• Real-Time Messaging with Hybrid-State Architectures: Optimizing Latency and Cost in Generative AI-Enhanced Real-Time Messaging at the Edge — 2026 · International Journal of Computational ... adaptive
• Rethinking Speculative Decoding: Distributed Inference Roles for Quality-Latency Tradeoffs under LLM-as-a-Judge Evaluation — 2026
• SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation — 2026 · arXiv
• The Disparate Impacts of Speculative Decoding — 2025 · arXiv
• ULTIMATE GENESIS AI v7.4 — 2026 · Zenodo (CERN European Organization for ...
• WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inference — 2026 · arXiv

KV Cache, Offloading & Quantization

How speculation composes with the other memory optimisations.

Reading picks

• ⭐ SpecExec: Massively Parallel Speculative Decoding for Interactive LLM Inference on Consumer Devices
• ⭐⭐⭐ QSpec: Speculative Decoding with Complementary Quantization Schemes

More papers in this category

• Disaggregated Speculative Decoding for Carbon-Efficient LLM Serving — 2025 · IEEE Computer Architecture Letters
• AutoChemSchematic AI: Agentic Physics-Aware Automation for Chemical Manufacturing Scale-Up — 2025 · arXiv
• 31.1 A 14.08-to-135.69Token/s ReRAM-on-Logic Stacked Outlier-Free Large-Language-Model Accelerator with Block-Clustered Weight-Compression and Adaptive Parallel-Speculative-Decoding — 2026 · arXiv adaptive
• AdaptiveSD A Stability-Aware, Runtime-Adaptive Speculative Decoding Framework with Multi-Policy Orchestration for CPU-Constrained LLM Inference — 2026 · arXiv adaptive
• An Adaptive Parallel Layer-Skipping Framework for Large Language Model Inference Speedup With Speculative Decoding — 2025 · Integrated Circuits and Systems adaptive
• BranchKV: Branch-Aware Copy-on-Write Memory Management for Speculative LLM Inference — 2026 · Open MIND
• CATS: Cascaded Adaptive Tree Speculation for Memory-Limited LLM Inference Acceleration — 2026 · arXiv adaptive
• Causal Topologies for Edge Computing: Intrinsic Efficiency and Deep Intelligence Beyond the Transformer Paradigm — 2026 · Zenodo (CERN European Organization for ... adaptive
• Communication-Efficient Speculative Decoding for Large Language Models Inference — 2025
• Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing — 2026 · arXiv
• Do we really Need More Parameters, or the Right Parameters? A Comparative Study of AI Text Humanization Across Model Scales — 2026 · Zenodo (CERN European Organization for ...
• Draft-based Approximate Inference for LLMs — 2025 · arXiv
• Entropy-Guided Dynamic Expert Selection in Mixture-of-Experts Models — 2026 · Zenodo (CERN European Organization for ... adaptive
• Ep. 267: Decoding the Transformer: From Attention to Inference — 2026 · Zenodo (CERN European Organization for ...
• Ep. 633: Memory Wars: The Future of Local Agentic AI — 2026 · Zenodo (CERN European Organization for ...
• HAMburger: Accelerating LLM Inference via Token Smashing — 2025 · arXiv
• HYBRID METHODS FOR STABILIZING THE GENERATION OF STRUCTURED DATA BY SMALL LANGUAGE MODELS IN INTERACTIVE REAL-TIME SYSTEMS — 2026 · Municipal economy of cities adaptive
• INFERENCE ENGINE — A Creative Research Document — 2026 · Zenodo (CERN European Organization for ...
• Inference-Time Efficiency as a Structural Variable: How Commitment Boundaries, KV Cache Economics, Speculative Verification, Reward-Channel Addiction, and Deliberative Hybridization Jointly Constrain the Design Space of Scalable Language Model Deployment — 2026 · Zenodo (CERN European Organization for ...
• LLMs on a Budget? Say HOLA — 2025 · Open MIND
• Leveraging Speculative Sampling and KV-Cache Optimizations Together for Generative AI using OpenVINO — 2023 · arXiv adaptive
• Local-First AI Inference: Architectural Patterns for Fully Offline LLM Deployment — API Gateway, AI Routing, Distributed Systems, Sovereign AI, and Post-Cloud Architecture (Api-Oss-Fixed) — 2026 · Zenodo (CERN European Organization for ... adaptive
• MAXTOKEN A Unified Framework for Unbounded Output Generation and Repository-Scale Code Understanding — 2026 · Zenodo (CERN European Organization for ... adaptive
• MiniCPM4: Ultra-Efficient LLMs on End Devices — 2025 · arXiv
• MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios — 2026 · arXiv adaptive
• Multi-Tenant Edge-Cloud Hybrid LLM Serving using Speculative Decoding and Quantization — 2026 · Research Square
• Proofread: Fixes All Errors with One Tap — 2024 · arXiv
• QuickSilver -- Speeding up LLM Inference through Dynamic Token Halting, KV Skipping, Contextual Token Fusion, and Adaptive Matryoshka Quantization — 2025 · arXiv adaptive
• Real-time RTF on consumer GPUs — 2026 · Zenodo (CERN European Organization for ...
• SLM-Fusion: A Unified Framework for Model Merging, Routing, and Multi-Model Orchestration — 2025 adaptive
• SPIRe: Boosting LLM Inference Throughput with Speculative Decoding — 2025 · arXiv
• SpecAttn: Co-Designing Sparse Attention with Self-Speculative Decoding — 2026 · arXiv lossless
• SpecAttn: Speculating Sparse Attention — 2025 · arXiv training-free adaptive
• Spectra: An Asymmetric-Precision Speculative Decoding LLM Accelerator with Product Quantization and Reconfigurable Flip-Flop Buffers in 28Nm CMOS — 2026
• Speculative Decoding Meets Quantization: Compatibility Evaluation and Hierarchical Framework Design — 2025 · arXiv
• Streaming-dLLM: Accelerating Diffusion LLMs via Suffix Pruning and Dynamic Decoding — 2026 · Open MIND training-free adaptive
• Systematic Evaluation of Evaluation Protocol Factors Explaining Divergent Qwen2.5 Performance on the Ruler Benchmark — 2026 · Zenodo (CERN European Organization for ... adaptive
• The Speed of Thought: Inside the New Era of Inference — 2026 · Zenodo (CERN European Organization for ...
• Vegas: Self-Speculative Decoding with Verification-Guided Sparse Attention — 2026 · Open MIND lossless
• Whisper-Flash: Continuous Manifold Decoding and the Branching Ceiling in Speculative ASR — 2026 · Zenodo (CERN European Organization for ... adaptive

On-Device & Edge

Where the memory-bound argument is strongest.

Reading picks

• ⭐ LLMCad: Fast and Scalable On-device Large Language Model Inference

More papers in this category

• EdgeLLM: Fast On-Device LLM Inference With Speculative Decoding — 2024 · IEEE Transactions on Mobile Computing tree adaptive
• Communication-Efficient Collaborative LLM Inference via Distributed Speculative Decoding — 2025
• MatFormer: Nested Transformer for Elastic Inference — 2023 · arXiv adaptive
• Edge-side NPU inference optimization: Adaptation research of multimodal large models on qualcomm platforms — 2025 · Intelligent Data Analysis
• A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference — 2026 · arXiv training-free
• AHASD: Asynchronous Heterogeneous Architecture for LLM Adaptive Drafting Speculative Decoding on Mobile Devices — 2026 · arXiv adaptive
• Accelerating Inference in Lightweight Language Models Using Speculative Decoding and Quantization — 2025
• Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Execution — 2025 · arXiv
• Agent-X: Full Pipeline Acceleration of On-device AI Agents — 2026 · arXiv
• An Empirical Study of Speculative Decoding for Small Language Models — 2026 · Underline Science Inc.
• Area- and Utilization-Efficient LLM Accelerator With Fused Speculative Decoding for Edge-Side Inference — 2026 · IEEE Transactions on Very Large Scale I...
• Collaborative Edge Inference for Large Language Models with Speculative Decoding — 2026
• Collaborative Large Language Model Inference via Resource-Aware Parallel Speculative Decoding — 2025 · arXiv
• Collaborative Multi-Device Edge Inference for Vision-Language Models with Speculative Decoding — 2025
• Compiler-Assisted Speculative Sampling for Accelerated LLM Inference on Heterogeneous Edge Devices — 2026 · Open MIND
• Edge-Cloud Collaborative Speech Emotion Captioning via Token-Level Speculative Decoding in Audio-Language Models — 2026 · arXiv
• Efficient LLM Inference Over Heterogeneous Edge Networks With Speculative Decoding — 2026 · IEEE Transactions on Communications
• Efficient On-Device Diffusion LLM Inference with Mobile NPU — 2026 · arXiv
• Efficient and Privacy Aware Edge Cloud Collaborative Inference for Large Language Models — 2026 · arXiv adaptive
• Energy-Efficient Wireless LLM Inference via Uncertainty and Importance-Aware Speculative Decoding — 2025 · arXiv
• Ep. 477: Can Your Phone Actually Think Without the Cloud? — 2026 · Zenodo (CERN European Organization for ...
• Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exits — 2025 · arXiv
• FiRST: Finetuning Router-Selective Transformers for Input-Adaptive Latency Reduction — 2024 · Open MIND adaptive
• GELATO: Generative Entropy- and Lyapunov-based Adaptive Token Offloading for Device-Edge Speculative LLM Inference — 2026 · arXiv adaptive
• Ghidorah: Fast LLM Inference on Edge with Speculative Decoding and Hetero-Core Parallelism — 2025 · arXiv
• Knowledge Distillation for Large Language Models: A Survey with a Case Study for Human-Robot Interaction and Edge Deployment — 2026
• Lever: Speculative LLM Inference on Smartphones — 2026 · arXiv
• LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environments — 2025 · arXiv
• Memory-Efficient Speculative Decoding for Large Language Model Inference on Commodity GPUs — 2026
• Model-Free Speculative Decoding for Transformer-Based ASR with Token Map Drafting — 2025
• PELM: Power Efficient On-Device LLM Inference with Speculative Decoding and Dynamic Voltage Frequency Scaling — 2026 adaptive
• Prada: Black-Box LLM Adaptation with Private Data on Resource-Constrained Devices — 2025 · arXiv
• Spec-LLaVA: Accelerating Vision-Language Models with Dynamic Tree-Based Speculative Decoding — 2025 · arXiv lossless tree adaptive
• SpecMemo: Speculative Decoding is in Your Pocket — 2025 · arXiv
• WiP: Efficient Speculative Decoding for AI PCs via Hierarchical N-Gram Retrieval — 2025

Production Implementations

vLLM, TensorRT-LLM, SGLang, llama.cpp, and friends.

Implementation-first reading

This branch changes faster than papers can track it. Start with the current vLLM, TensorRT-LLM, SGLang, llama.cpp, or Hugging Face documentation for your stack, then use the papers below for implementation case studies.

More papers in this category

• Breaking the Reward Barrier: Accelerating Tree-of-Thought Reasoning via Speculative Exploration — 2026 · arXiv tree adaptive
• Dynamic K Mechanism for Accelerating Speculative Decoding in vLLM — 2025 adaptive
• Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving — 2026 · arXiv
• Guess-Verify-Refine: Data-Aware Top-K for Sparse-Attention Decoding on Blackwell via Temporal Correlation — 2026 · arXiv
• Hayula Research Paper — 2026 · Zenodo (CERN European Organization for ...
• Local LLM Inference for Privacy-Preserving Browser Intelligence: Architecture and Optimization — Browser Engine, Privacy, Web, Sovereign AI, and Post-Cloud Architecture (Kathon) — 2026 · Zenodo (CERN European Organization for ...
• MineDraft: A Framework for Batch Parallel Speculative Decoding — 2026 · arXiv
• SpecForge: A Flexible and Efficient Open-Source Training Framework for Speculative Decoding — 2026 · arXiv
• Trees from Marginals: Autoregressive drafting with factorized priors — 2026 · arXiv
• When RL Meets Adaptive Speculative Training: A Unified Training-Serving System — 2026 · Open MIND adaptive

Applications & Modalities

Long Context

Where speculation becomes attractive again at large batch.

Reading picks

• ⭐⭐ TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding
• ⭐⭐⭐ MagicDec: Breaking the Latency-Throughput Tradeoff for Long Context Generation with Speculative Decoding

More papers in this category

• Accelerate Speculative Decoding with Sparse Computation in Verification — 2025 · arXiv
• BudgetDraft: Acceptance-Aware Multi-View Training for Sparse-KV Speculative Decoding — 2026 · arXiv
• KVBuffer: IO-aware Serving for Linear Attention — 2026 · arXiv
• KnapSpec: Self-Speculative Decoding via Adaptive Layer Selection as a Knapsack Problem — 2026 · Open MIND training-free adaptive
• OWL: Overcoming Window Length-Dependence in Speculative Decoding for Long-Context Inputs — 2025 · arXiv
• SpecPV: Improving Self-Speculative Decoding for Long-Context Generation via Partial Verification — 2025 · arXiv
• SpecPrefill: Speculative Sparse Prefill for Efficient Long-Context LLM Inference on Apple Silicon — 2026 · Open MIND

Multimodal, Speech & Video

Reading picks

• ⭐ Faster Speech-LLaMA Inference with Multi-token Prediction

More papers in this category

• SEC : Enabling MLLMs for Low-Latency IoT Video Analysis via Semantic-Aware Edge–Cloud Collaboration — 2026 · IEEE Internet of Things Journal tree adaptive
• Adaptive Two-Layer Inspection Framework for Mitigating Security Risks in Large-Scale Vertical Domain Language Models — 2025 adaptive
• Algorithms and Systems for Efficient Inference in Generative AI — 2026 · Underline Science Inc.
• AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression — 2026 · Open MIND training-free adaptive
• Annealed Relaxation of Speculative Decoding for Faster Autoregressive Image Generation — 2026 · Proceedings of the AAAI Conference on A...
• Breaking Block Boundaries: Anchor-based History-stable Decoding for Diffusion Large Language Models — 2026 · Underline Science Inc. training-free adaptive
• CSD: Content-aware Speculative Decoding for Efficient Image Generation — 2026 · arXiv
• CoSense-LLM: Semantics at the Edge with Cost- and Uncertainty-Aware Cloud-Edge Cooperation — 2025 · arXiv
• Continuous Speculative Decoding for Autoregressive Image Generation — 2024 · arXiv
• Ep. 132: Can AI Map Your House Just by Looking Around? — 2026 · Zenodo (CERN European Organization for ...
• Fast Inference of Visual Autoregressive Model with Adjacency-Adaptive Dynamical Draft Trees — 2025 · arXiv tree adaptive
• FastV-RAG: Towards Fast and Fine-Grained Video QA with Retrieval-Augmented Generation — 2026 · Underline Science Inc.
• Faster Than Thought: The Engineering Behind Real-Time AI — 2026 · Zenodo (CERN European Organization for ...
• Faster and Better LLMs via Latency-Aware Test-Time Scaling — 2025 · Open MIND
• Grouped Speculative Decoding for Autoregressive Image Generation — 2025 · arXiv
• H2E: Human-to-Expert AI Governance Through Geometric Safety Manifolds Technical Status Report & Roadmap — UNESCO Resilient AI Challenge — 2026 · Zenodo (CERN European Organization for ...
• HIPPO: Accelerating Video Large Language Models Inference via Holistic-aware Parallel Speculative Decoding — 2026 · arXiv
• HiViS: Hiding Visual Tokens from the Drafter for Speculative Decoding in Vision-Language Models — 2025 · arXiv
• KERV: Kinematic-Rectified Speculative Decoding for Embodied VLA Models — 2026 · arXiv
• Multi-Scale Local Speculative Decoding for Image Generation — 2026 · arXiv
• Non-autoregressive speech translation with understanding, translation, reordering and LLM-augmented correction — 2026 · APSIPA Transactions on Signal and Infor... training-free
• On Speculative Decoding for Multimodal Large Language Models — 2024 · arXiv
• PredGen: Accelerated Inference of Large Language Models through Input-Time Speculation for Real-Time Speech Interaction — 2025 · arXiv
• Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving — 2026 · arXiv
• SSD: Spatially Speculative Decoding Accelerates Autoregressive Image Generation — 2026 · arXiv
• See the Forest for the Trees: Loosely Speculative Decoding via Visual-Semantic Guidance for Efficient Inference of Video LLMs — 2026 · Underline Science Inc. training-free
• Small Drafts, Big Verdict: Information-Intensive Visual Reasoning via Speculation — 2025 · arXiv training-free
• Sparrow: Text-Anchored Window Attention with Visual-Semantic Glimpsing for Speculative Decoding in Video LLMs — 2026 · Underline Science Inc.
• SpeCa: Accelerating Diffusion Transformers with Speculative Feature Caching — 2025 · arXiv adaptive
• SpecFLASH: A Latent-Guided Semi-autoregressive Speculative Decoding Framework for Efficient Multimodal Generation — 2025 · arXiv
• SpecVLM: Fast Speculative Decoding in Vision-Language Models — 2025 · arXiv lossless adaptive
• Speculative Decoding Reimagined for Multimodal Large Language Models — 2025 · arXiv
• Speculative Decoding for Autoregressive Video Generation — 2026 · arXiv training-free
• Speculative Decoding for Multimodal Models: A Survey — 2026 · Preprints.org
• ViDA: Lossless VideoQA Acceleration via Selective Sparse Self-Speculation With Parallel Computational Load Management — 2026 · IEEE Transactions on Parallel and Distr... lossless adaptive

Reasoning & Test-Time Compute

Reading picks

• ⭐ Reward-Guided Speculative Decoding for Efficient LLM Reasoning

More papers in this category

• SAIL: Redesigning Collaborative Language Inference with a Single Server-to-Mobile Handoff — 2026 adaptive
• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts — 2025 · arXiv
• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction — 2025 · arXiv adaptive
• Aggressive Speculative Decoding for Efficient Chain-of-Thought Reasoning — 2025 · Tsinghua Science & Technology
• Beat the long tail: Distribution-Aware Speculative Decoding for RL Training — 2025 · arXiv adaptive
• Beyond Tokens: Semantic-Aware Speculative Decoding for Efficient Inference by Probing Internal States — 2026 · Open MIND
• CoTu at EXACT 2026: Neuro-Symbolic Reasoning for Transparent Educational QA — 2026 · arXiv
• Democratizing LLM Efficiency: From Hyperscale Optimizations to Universal Deployability — 2025 · Proceedings of the AAAI Conference on A... adaptive
• Free Draft-and-Verification: Toward Lossless Parallel Decoding for Diffusion Large Language Models — 2025 · arXiv lossless
• Guided Speculative Inference for Efficient Test-Time Alignment of LLMs — 2025 · arXiv
• HyperDFlash: Hyper-Connection-Aligned Block Speculative Decoding with Gated Residual Reduction — 2026 · arXiv
• Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair — 2026 · arXiv
• Interpretable Contrastive Monte Carlo Tree Search Reasoning — 2024 · arXiv
• LLM Foundations Applied: Optimizing Large Language Models for Apple M2 Ultra Consumer Hardware — 2026 · Zenodo (CERN European Organization for ...
• Less Experts, Faster Decoding: Cost-Aware Speculative Decoding for Mixture-of-Experts — 2026 · arXiv adaptive
• LongSpec: Long-Context Lossless Speculative Decoding with Efficient Drafting and Verification — 2026 · Underline Science Inc. lossless tree
• Look Before You Leap: A Lookahead Reasoning Quality Gate for Speculative Decoding — 2026
• Mellum2 Technical Report — 2026 · arXiv
• Opus 4.8: What Actually Changed Under the Hood — 2026 · Open MIND adaptive
• PSD: Pushing the Pareto Frontier of Diffusion LLMs via Parallel Speculative Decoding — 2026 · arXiv training-free adaptive
• RelayGen: Intra-Generation Model Switching for Efficient Reasoning — 2026 · Open MIND training-free
• Revisiting Judge Decoding from First Principles via Training-Free Distributional Divergence — 2026 · arXiv training-free
• Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level — 2024 · arXiv
• Root Defense Strategies: Ensuring Safety of LLM at the Decoding Level — 2025
• S$^4$C: Speculative Sampling with Syntactic and Semantic Coherence for Efficient Inference of Large Language Models — 2025 · arXiv
• SEED: Accelerating Reasoning Tree Construction via Scheduled Speculative Decoding — 2024 · arXiv training-free
• SLOs-Serve: Optimized Serving of Multi-SLO LLMs — 2025 · arXiv adaptive
• SPEC-RL: Accelerating On-Policy Reinforcement Learning with Speculative Rollouts — 2025 · arXiv
• SSR: Speculative Parallel Scaling Reasoning in Test-time — 2025 · arXiv training-free
• Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling — 2025 · arXiv
• Sharded Inference of a 229B-Parameter MoE over the Public Internet at Interactive Speed — 2026 · Zenodo (CERN European Organization for ...
• Sherlock: Reliable and Efficient Agentic Workflow Execution — 2025 · arXiv
• SpecCoT: Accelerating Chain-of-Thought Reasoning through Speculative Exploration — 2025 · Open MIND
• SpecExit: Accelerating Large Reasoning Model via Speculative Exit — 2025 · arXiv
• SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning — 2025 · arXiv
• Speculative Thinking: Enhancing Small-Model Reasoning with Large Model Guidance at Inference Time — 2025 · arXiv training-free
• T-pro 2.0: An Efficient Russian Hybrid-Reasoning Model and Playground — 2025 · Underline Science Inc.
• TAPS: Task Aware Proposal Distributions for Speculative Sampling — 2026 · arXiv tree
• The honest machine proposing a hybrid verification model for AI-assisted academic research — 2026 · The International Islamic University Ma...

Agents, Tool Use & Structured Output

Reading picks

• ⭐ Speculative Actions: A Lossless Framework for Faster Agentic Systems

More papers in this category

• ARC: Decode-Time Behavioral Control and Guarded Self-Optimization in Large Language Models — 2026 · Zenodo (CERN European Organization for ... adaptive
• Cache Hit Rate as a Primary Cost Predictor in Production LLM Systems: Context Complexity Over Task Complexity — 2026 · Zenodo (CERN European Organization for ...
• Hybrid Verified Decoding: Learning to Allocate Verification in Speculative Decoding — 2026 · arXiv
• Precompiled KV Memory Modules: Relocatable, Composable Agent Memory for LLM Inference Runtimes — 2026 · Zenodo (CERN European Organization for ...
• Scaling Embeddings Outperforms Scaling Experts in Language Models — 2026 · Open MIND

Retrieval-Augmented Generation

Reading picks

• ⭐ Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting

More papers in this category

• Speculative Retrieval-Augmented Generation for Cost-Efficient Large Language Model Inference — 2026 · International Scientific Journal of Eng...

Diffusion & Non-Autoregressive Models

Reading picks

• ⭐ Speculative Diffusion Decoding: Accelerating Language Generation through Diffusion

More papers in this category

• DiffuSpec: Unlocking Diffusion Language Models for Speculative Decoding — 2026 · arXiv adaptive
• Diffusion Language Model Parallel Decoding via Product-of-Experts Bridge — 2026 · arXiv
• Dynamic-dLLM: Dynamic Cache-Budget and Adaptive Parallel Decoding for Training-Free Acceleration of Diffusion LLM — 2026 · arXiv training-free adaptive
• Factorization-Error-Free Discrete Diffusion Language Model via Speculative Decoding — 2026 · arXiv
• Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding — 2025 · arXiv training-free
• Inverse Speculation: Structural Anchoring from Diffusion Language Models for Edge-Scale Generation — 2026 · Open MIND
• Rejection Mixing: Fast Semantic Propagation of Mask Tokens for Efficient DLLM Inference — 2026 · Open MIND training-free
• Reviving Any-Subset Autoregressive Models with Principled Parallel Sampling and Speculative Decoding — 2025 · arXiv
• SpecDiff-2: Scaling Diffusion Drafter Alignment For Faster Speculative Decoding — 2025 · arXiv lossless
• Structuring The Future: Diffusion LLM Speculative Decoding via Calibrated Draft Graphs — 2025 · arXiv adaptive
• TEAM: Temporal-Spatial Consistency Guided Expert Activation for MoE Diffusion Language Model Acceleration — 2026 · Open MIND training-free

Mixture-of-Experts Targets

Reading picks

• ⭐⭐ EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

More papers in this category

• D-cut: Adaptive Verification Depth Pruning for Batched Speculative Decoding — 2026 · arXiv adaptive
• Duo-LLM: A Framework for Studying Adaptive Computation in Large Language Models — 2024 · arXiv adaptive
• Lyapunov Stability Envelopes for Speculative Decoding — Rejection-Rate Compliance for Deterministic AI Systems — 2026 · Zenodo (CERN European Organization for ...
• Streaming Mixture-of-Experts Inference on Consumer Apple Silicon: A Measured Map of What Works and What Doesn't — 2026 · Zenodo (CERN European Organization for ...

Domain-Specific Applications

Translation, code, biomedical, recommendation.

Reading picks

• ⭐ FastDraft: How to Train Your Draft

More papers in this category

• Alignment-Augmented Speculative Decoding with Alignment Sampling and Conditional Verification — 2025 · Open MIND training-free adaptive
• Confidence-Modulated Speculative Decoding for Large Language Models — 2025 · arXiv

Theory & Evaluation

Speedup Models & Analysis

Expected acceptance length, optimal draft length.

Reading picks

• ⭐⭐ The Synergy of Speculative Decoding and Batching in Serving Large Language Models

More papers in this category

• Diffusion Models are Secretly Exchangeable: Parallelizing DDPMs via Autospeculation — 2025 · arXiv
• GRIFFIN: Effective Token Alignment for Faster Speculative Decoding — 2025 · arXiv
• Speculative Decoding: Performance or Illusion? — 2025 · arXiv
• Speeding up Speculative Decoding via Sequential Approximate Verification — 2025 · arXiv
• Think Before You Accept: Semantic Reflective Verification for Faster Speculative Decoding — 2025 · arXiv training-free

Optimality & Limits

Reading picks

• ⭐ SpecTr: Fast Speculative Decoding via Optimal Transport
• ⭐⭐⭐ Multi-Draft Speculative Sampling: Canonical Decomposition and Theoretical Limits

More papers in this category

• Coupling without Communication and Drafter-Invariant Speculative Decoding — 2024 · arXiv
• Towards Optimal Multi-draft Speculative Decoding — 2025 · arXiv

Benchmarks & Measurement

Wall-clock versus block-efficiency, and how to compare fairly.

Reading picks

• ⭐ Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding
• ⭐⭐⭐ Decoding Speculative Decoding

More papers in this category

• Accelerating PayPal's Commerce Agent with Speculative Decoding: An Empirical Study on EAGLE3 with Fine-Tuned Nemotron Models — 2026 · arXiv
• Acceptance Dynamics Across Cognitive Domains in Speculative Decoding — 2026 · arXiv tree
• An Empirical Study of Speculative Decoding on Software Engineering Tasks — 2026 · arXiv lossless
• Balancing Latency and Accuracy of Code Completion via Local-Cloud Model Cascading — 2026 · Open MIND
• Entropy Adaptive Decoding: Dynamic Model Switching for Efficient Inference — 2025 · arXiv adaptive
• Parallel Token Prediction for Language Models — 2025 · Open MIND
• Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafters — 2026 · arXiv
• Uncertainty-Aware Speculative Decoding for Diffusion Language Models in Long Document Generation — 2026 · Jisuanji shenghuojia.

Statistics

Papers per year (by first public appearance):

Year	Papers
2018	1	█
2022	2	█
2023	16	██
2024	113	████████████
2025	244	██████████████████████████
2026 (partial)	369	████████████████████████████████████████

The corpus snapshot ends 2026-07-25, so 2026 is a partial year and is further under-counted by catalogue indexing lag — treat its row as a lower bound.

Papers per taxonomy branch:

Category	Papers
survey — Surveys & Position Papers	18
verify-lossless — Lossless Acceptance	34
draft-independent — Independent Draft Models	120
draft-self — Self-Drafting	37
draft-heads — Auxiliary Prediction Heads	52
draft-retrieval — Retrieval, N-gram & Model-Free	22
draft-mtp — Multi-Token Prediction Objectives	27
draft-cascade — Cascades & Hierarchies	7
verify-tree — Token-Tree Verification	36
verify-relaxed — Relaxed & Lossy Acceptance	12
verify-multidraft — Multi-Draft & Optimal Acceptance	5
train-distill — Distillation	19
train-online — Online & Test-Time Adaptation	5
train-arch — Drafter Architecture	1
sys-batching — Batching, Scheduling & Throughput	121
sys-distributed — Distributed & Pipelined Serving	13
sys-memory — KV Cache, Offloading & Quantization	42
sys-edge — On-Device & Edge	36
sys-impl — Production Implementations	10
app-longcontext — Long Context	9
app-multimodal — Multimodal, Speech & Video	36
app-reasoning — Reasoning & Test-Time Compute	40
app-agents — Agents, Tool Use & Structured Output	6
app-rag — Retrieval-Augmented Generation	2
app-diffusion — Diffusion & Non-Autoregressive Models	12
app-moe — Mixture-of-Experts Targets	4
app-domain — Domain-Specific Applications	3
theory-analysis — Speedup Models & Analysis	5
theory-optimal — Optimality & Limits	3
eval-benchmark — Benchmarks & Measurement	8

Regenerate with make stats (writes figures to paper/figures/).

License

Apache-2.0 for the code; the bibliography is factual metadata.