Alibaba's Qwen3-ASR Beats GPT-4o Transcription Running Locally on 1.32 GB
Qwen3-ASR-1.7B is now available as GGUF quantizations for transcribe.cpp local inference. Six variants from BF16 (4.08 GB) down to Q4_K_M (1.32 GB) with WER staying between 1.62% and 1.81%. Supports 52 languages and dialects , including 22 Chinese dialects, Cantonese, and singing voice.
Takeaways
- Qwen3-ASR-1.7B is now available as GGUF quantizations for transcribe.cpp local inference.
- Six variants from BF16 (4.08 GB) down to Q4_K_M (1.32 GB) with WER staying between 1.62% and 1.81%.
- Supports 52 languages and dialects , including 22 Chinese dialects, Cantonese, and singing voice.
- Beats Whisper-large-v3 on most benchmarks and matches GPT-4o-Transcribe on several.
- Base model from Qwen is Apache-2.0, free for commercial use.
- Needs 16 kHz mono WAV input; convert with ffmpeg before running the CLI.
A community port has packaged Alibaba’s 1.7-billion-parameter Qwen3-ASR speech recognizer as six GGUF files for local transcription. The GGUF repository targets transcribe.cpp , a native C++ runtime modeled on llama.cpp that can operate without a discrete GPU.
The files range from about 1.32 GB to 4.08 GB and support automatic language identification across 30 languages and 22 Chinese dialects. The Hugging Face page displayed more than 897,000 downloads when this article was prepared. That counter records file pulls, including possible automated traffic.
Six files, one native runtime
The release converts the original base weights into GGUF and validates them against a specific transcribe.cpp revision listed on the model card. Pinning that revision improves reproducibility because runtime loaders and model metadata can change.
GGUF stores model tensors, tokenizer data, and configuration metadata in a format that native runtimes can memory-map. Quantization reduces the precision of model weights, cutting file size and memory bandwidth at a possible cost to recognition accuracy.
Each build accepts 16 kHz mono audio, identifies the language, and generates text. The underlying architecture combines a bidirectional audio encoder with a Qwen3 causal language model, injecting encoded audio features into the text model before decoding the transcript.
| Build | Approximate size | WER |
|---|---|---|
| BF16 or F16, two files | 4.08 GB each | 1.62% |
| Q8_0 | 2.19 GB | 1.62% |
| Q6_K | 1.69 GB | 1.65% |
| Q5_K_M | 1.52 GB | 1.65% |
| Q4_K_M | 1.32 GB | 1.81% |
The port’s authors evaluated the full LibriSpeech test-clean split on an Apple M4 with the Whisper English normalizer and jiwer 3.x. Word error rate, or WER, measures substitutions, deletions, and insertions relative to a reference transcript; lower values indicate fewer errors.
The model card’s bootstrap analysis places BF16, F16, Q8_0, Q6_K, and Q5_K_M within the same confidence range. Q4_K_M adds 0.19 absolute WER points while reducing the file to 1.32 GB. These results cover clean English speech and report accuracy rather than decoding speed, peak RAM, or real-time factor. Runtime buffers and context require memory beyond the GGUF file itself.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
Read original report
