Unofficial Qwen3.8-27B Retrofit Runs 2.92x Faster at 261K Tokens
Community retrofit adds Qwen Sparse Attention to Qwen3.8-27B, previewing a core Qwen4 technique on an existing checkpoint. At 261k tokens, QSA delivers 2.9x faster prefill and 1.8x faster decode than dense.
Takeaways
- Community retrofit adds Qwen Sparse Attention to Qwen3.8-27B, previewing a core Qwen4 technique on an existing checkpoint.
- At 261k tokens, QSA delivers 2.9x faster prefill and 1.8x faster decode than dense.
- Uses a small indexer scoring 4-token blocks with a fixed 2048-token attention budget per query.
- BF16 perplexity ratio vs dense is 0.99978 on 32k tokens; broad quality unverified.
- Ships as Q8_0 GGUF plus a patched llama.cpp runtime ; stock llama.cpp cannot run it.
- Prebuilt binary targets Linux x86_64 and NVIDIA sm70; other hardware needs source build.
An independent developer has released an unofficial Qwen Sparse Attention retrofit for Qwen3.8-27B. The model repository contains a quantized multimodal checkpoint that converts the model’s 16 full-attention layers to QSA. In the author’s test at 261,120 input tokens, it processed prompts 2.92 times as fast and generated tokens 1.79 times as fast as the dense baseline. Two narrow BF16 checks produced effectively unchanged results.
The release packages 29.3 GB of Q8_0 weights and a prebuilt runtime from a patched llama.cpp fork . Upstream llama.cpp lacks the required QSA execution path. The author describes the project as an early implementation of a technique expected to appear in Qwen4.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
