APIUp to 25% cheaper than official pricesTry the API →
Hermes
APIModelsCases
Back to News
paperLlmsTraining Infra

Tsinghua's SMELT Cuts AI Training Costs 18% by Looping Layers Twice

New scaling laws show that looping the middle half of a Mixture-of-Experts model twice saves up to 18% of training compute at matched budgets.

Tsinghua's SMELT Cuts AI Training Costs 18% by Looping Layers Twice
Source
AlphaSignal
Published
Author
AlphaSignal Newsroom
Read
1 min read

New scaling laws show that looping the middle half of a Mixture-of-Experts model twice saves up to 18% of training compute at matched budgets.

Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.

Read original report

Next reads

OT-RED Merges Distributed AI Models Across Hospitals With One Upload
AlphaSignal · paper

OT-RED Merges Distributed AI Models Across Hospitals With One Upload

Sewon Min's Preprint Shows Smaller Batches Beat Overfitting on Repeated Training Data
AlphaSignal · paper

Sewon Min's Preprint Shows Smaller Batches Beat Overfitting on Repeated Training Data

MSAR Cuts Transformer Training Compute by 37% With Smarter Residual Connections
AlphaSignal · paper

MSAR Cuts Transformer Training Compute by 37% With Smarter Residual Connections