Search Spins

The Convergence Behavior of Adam under Heavy-Tailed Noise

A new theoretical analysis establishes the first convergence guarantees for the standard Adam optimizer under heavy-tailed stochastic noise — a common but poorly understood condition in modern deep learning — revealing both its robustness and suboptimal iteration complexity without domain-radius adaptation.