prox-linear method

Some Unified Theory for Variance Reduced Prox-Linear Methods

  • Yue Wu
  • Benjamin Grimmer
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    This work considers the nonconvex, nonsmooth problem of minimizing a composite objective of the form $f(g(x))+h(x)$ where the inner mapping $g$ is a smooth finite summation or expectation amenable to variance reduction. In such settings, prox-linear methods can enjoy variance-reduced speed-ups despite the existence of nonsmoothness. We provide a unified convergence theory applicable to a … Read more

    Stochastic model-based minimization of weakly convex functions

  • Damek Davis
  • Dmitriy Drusvyatskiy
    • Categories Convex and Nonsmooth Optimization Tags , , , , ,

    We consider an algorithm that successively samples and minimizes stochastic models of the objective function. We show that under weak-convexity and Lipschitz conditions, the algorithm drives the expected norm of the gradient of the Moreau envelope to zero at the rate $O(k^{-1/4})$. Our result yields the first complexity guarantees for the stochastic proximal point algorithm … Read more