Kakao Trains a 155B MoE on 10T Tokens Without a Single Hyperparameter Sweep
Kakao researchers show how a two-step transfer trick predicts the optimal learning rate for a 10-trillion-token MoE run using tiny proxy models.
// Intelligence
Training Infra
A focused feed of models, agents, research and open-source releases for people building with AI.
Updated
No stories match these filters.