Nesterov Momentum Escapes AI Training's Worst Traps Almost Every Time
A new analysis settles whether momentum-based optimizers can get stuck at saddle points, and designs a variant that escapes faster without hurting convergence.
Source
AlphaSignal
Published
Author
AlphaSignal Newsroom
Read
1 min read
A new analysis settles whether momentum-based optimizers can get stuck at saddle points, and designs a variant that escapes faster without hurting convergence.
Reporting is indexed from AlphaSignal. Rights remain with the original publisher and cited sources.
