Optimal First-Order Methods for Nonconvex Optimization Using the Stochastic Path-Integrated Differential Estimator

In this paper, we address the challenging problem of optimizing nonconvex functions commonly encountered in large-scale statistical learning tasks. We introduce the \emph{Stochastic Path-Integrated Differential EstimatoR} (\texttt{SPIDER}), a novel technique designed to efficiently track deterministic quantities with significantly reduced sampling costs. By leveraging \texttt{SPIDER}, we develop an algorithm that achieves a faster rate of convergence … Read more