Sharpening Tax in Post-Training evaluated 14 base/post-trained model pairs across three agentic benchmarks. Post-training generally improved first-attempt success ( pass@1 ), while base models often achieved greater coverage across repeated attempts ( pass@K ).
Source
Akruti Acharya
Published
Author
Akruti Acharya
Read
1 min read
Takeaways
Sharpening Tax in Post-Training evaluated 14 base/post-trained model pairs across three agentic benchmarks. Post-training generally improved first-attempt success ( pass@1 ), while base models often achieved greater coverage across repeated attempts ( pass@K ).
On WebShop , base Gemma-4-31B reached 87.6% pass@128 , versus 56.0% for its post-trained counterpart. The base model overtook it after approximately three attempts.
A lightweight harness increased average pass@32 on BFCL from 15.19% to 49.13% across eight base models, but reduced average performance when applied to post-trained models.
Mid-Harness improved TMAX-9B success on TerminalBench-Lite from 50.00% to 68.03% by selecting among eight candidate actions using GPT-5.6 Sol , without retraining the generator.
In an AlphaSignal interview, Minki Kang said verification proved harder than expected. A distilled 9B verifier improved task success from 54.76% to 57.14% , but struggled with command semantics, environment state, and execution consequences.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
