APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

Inferact's Kimi K3.1 Pro DSpark Accepts 23% More Tokens, Cuts Draft Memory

Inferact and Red Hat released a new Kimi K3.1 Pro DSpark draft model that replaces the preview. Acceptance length is up to 30% higher on HumanEval and 23% higher pooled across benchmarks. Every draft layer uses a 2K sliding window, so draft KV memory stays fixed up to 1M-token context.

Inferact's Kimi K3.1 Pro DSpark Accepts 23% More Tokens, Cuts Draft Memory
Source
vLLM
Published
Author
vLLM
Read
5 min read

Takeaways

  • Inferact and Red Hat released a new Kimi K3.1 Pro DSpark draft model that replaces the preview.
  • Acceptance length is up to 30% higher on HumanEval and 23% higher pooled across benchmarks.
  • Every draft layer uses a 2K sliding window, so draft KV memory stays fixed up to 1M-token context.
  • At 1M context, acceptance rose from 2.77 to 3.42 compared with the full-attention preview.
  • Weights are free. Upgrading is a one-line --speculative-config change in vLLM, and the vLLM recipes include a preset.
  • Gains are weakest on chat and agentic coding (MT-Bench 3.39, SWE-bench Pro 3.30).

Inferact and the Red Hat AI Speculators team have released a new DSpark checkpoint for Kimi K3.1 Pro. It replaces Kimi-K3.1-Pro-DSpark-preview, accepts more speculative tokens across every shared benchmark, and bounds the draft model’s KV cache with a 2,048-token attention window.

A small draft model makes speculative decoding practical for Kimi K3.1 Pro, a 2.8-trillion-parameter mixture-of-experts model. The draft proposes several tokens, then the target model verifies the block in one forward pass. Longer accepted runs reduce the number of expensive target-model passes required to generate a response. Compatible rejection sampling preserves the target model’s sampling distribution.

BenchmarkPreviewNew checkpointChange
HumanEval3.5854.660+30%
GSM8K4.5975.566+21%
SWE-bench Pro2.7413.302+20%
BEAM 1M2.7653.421+24%
Pooled result3.1543.864+23%
WorkloadAcceptance lengthOutput pattern
GSM8K5.566Structured math reasoning
HumanEval4.660Short code generation
BEAM 1M3.421Long-context generation
MT-Bench3.393Open-ended conversation
SWE-bench Pro3.302Agentic software engineering

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01x.com
  2. 02huggingface.co
  3. 03huggingface.co
  4. 04vllm.ai
  5. 05huggingface.co
  6. 06recipes.vllm.ai