Liquid AI's LFM2.5 Hits 600 Tok/s but Fails Every Quality Check
Community LoRA fine-tune converts LFM2.5-24B-A2B into a block masked-diffusion sampler. Hits 601.95 output tokens/s median on a single A100 80GB with 12 denoising steps. Author warns outputs show severe repetition, mixed languages, malformed Python, and token loops.
Takeaways
- Community LoRA fine-tune converts LFM2.5-24B-A2B into a block masked-diffusion sampler.
- Hits 601.95 output tokens/s median on a single A100 80GB with 12 denoising steps.
- Author warns outputs show severe repetition, mixed languages, malformed Python, and token loops.
- Trained with LoRA rank 64 on Alpaca and ARC-Easy for 3,100 updates; merged into BF16.
- Masked-token NLL improved up to 10.39%, but no sampler setting passed basic quality checks.
- Useful reference for diffusion-LLM researchers; not recommended for production use.
A community checkpoint on Hugging Face converts Liquid AI’s mixture-of-experts reasoning model into a block masked-diffusion sampler. Published by summerAIO, diffusion-lfm2.5-24b-a2b is an unofficial LoRA-merged fine-tune that reports more than 600 output tokens per second on one A100. Its generations fail the author’s basic quality checks, making the release a documented experiment rather than a production model.
An MoE rewired for diffusion
The upstream checkpoint contains 24 billion parameters, with 2.45 billion active for each token. Its mixture-of-experts blocks contain 64 experts and route each token through the top four. The 40-layer architecture combines 30 double-gated short-convolution blocks with 10 grouped-query attention layers and supports a 131K-token context window.
SummerAIO trained a masked-diffusion LoRA adapter with rank 64 and alpha 128, then merged it into the BF16 base weights and packed the expert weights. LoRA updates a compact set of low-rank parameters, reducing fine-tuning costs compared with updating every model weight. Training used Alpaca and ARC-Easy data for 3,100 update steps, with the final checkpoint selected by validation masked-token negative log likelihood.
The merged artifact occupies about 48GB on disk. It relies on a custom modeling_diffusion_lfm.py implementation, so standard Transformers generation code will not run unchanged. Integrators need the repository’s sampler and should audit the custom modeling code before allowing a loader to execute it.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
