Princeton Researchers Fix Costly AI Training Mistakes With 100x Smaller Learning Rates

A new study shows that proxy models used to pick pretraining data can flip their verdicts with tiny learning rate changes, with a simple fix.