APIUp to 28% cheaper than official pricesTry the API →
Hermes
Back to News

Alibaba's Qwen2.5-Math-PRM-7B Catches Exactly Where Your Math AI Goes Wrong

Qwen released Qwen2.5-Math-PRM-7B , a 7B Process Reward Model scoring each reasoning step from 0 to 1. Fine-tuned from Qwen2.5-Math-7B-Instruct with a scalar head replacing the LM head. Beats open-source PRMs and GPT-4o on ProcessBench style error localization; still trails o1-mini.

Alibaba's Qwen2.5-Math-PRM-7B Catches Exactly Where Your Math AI Goes Wrong
Source
Qwen
Published
Author
Qwen
Read
2 min read

Takeaways

  • Qwen released Qwen2.5-Math-PRM-7B , a 7B Process Reward Model scoring each reasoning step from 0 to 1.
  • Fine-tuned from Qwen2.5-Math-7B-Instruct with a scalar head replacing the LM head.
  • Beats open-source PRMs and GPT-4o on ProcessBench style error localization; still trails o1-mini.
  • Trained with consensus filtering on Monte Carlo step labels, detailed in arXiv:2501.07301 .
  • Use via Transformers with \n\n step breaks and <extra_0> sentinel tokens for per-step rewards.
  • Ideal for Best-of-N verification, PRM-guided search, and reasoning RL in English and Chinese math.

Alibaba’s Qwen team has released Qwen2.5-Math-PRM-7B , a 7-billion-parameter process reward model that assigns a score to each step in a candidate math solution. Developers can use those scores to rank sampled answers, guide search, or provide step-level rewards during training.

The release joins a larger 72B process reward model and Qwen’s existing outcome reward model. The outcome model assigns one score to a complete solution. The process model exposes local signals that can identify where a derivation deteriorates, while a separate policy model handles generation.

A 7B verifier in one view

PropertyDetails
Primary roleScore individual steps in mathematical reasoning
InputA question and a marked, multi-step candidate solution
OutputOne positive-class score from 0 to 1 per marked step
Base modelQwen2.5-Math-7B-Instruct
Related modelsA 72B PRM and Qwen’s outcome reward model
LicenseDownloadable weights under the license terms
AdoptionThe Hub showed 166,575 downloads in the preceding month.

One score for every step

A process reward model receives a problem and a completed reasoning chain, then emits a score at each marked boundary. Each value estimates the model’s confidence that the preceding step is acceptable in context. In Qwen’s worked example, scores of [1.00, 0.19, 0.98, 1.00] identify the second step as the likely failure point.

  • Best-of-N ranking: Generate N candidate solutions, score each chain, and select the highest-ranked answer.
  • Tree search: Score partial derivations so weak branches can be pruned before further decoding.
  • Reinforcement learning: Use intermediate scores as denser rewards than a single end-of-response grade.
  • Feedback systems: Flag a likely faulty step for a tutor, grader, or review interface.

Sentinels make the API work

Qwen’s interface depends on two delimiters: double newlines separate the written steps, and

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Sources

  1. 01huggingface.co
  2. 02arxiv.org
  3. 03huggingface.co
  4. 04emergentmind.com