Yue Wu

A Parameter-Free Restart Scheme with Only a Parallelizable $\log\log(1/\epsilon)$ Overhead

  • Yue Wu
  • Benjamin Grimmer
    • Categories Convex Optimization, Nonsmooth Optimization Tags , , ,

    It is well-known that first-order methods can offer accelerated convergence rates in the presence of growth structures. Restarting schemes provide a general tool for such speed-ups. These schemes typically either require unrealistic problem knowledge, incur logarithmic overhead factors in oracle complexity, and/or have a nontrivial initial burn-in phase. We present a parameter-free approach for restarting … Read more

    Some Unified Theory for Variance Reduced Prox-Linear Methods

  • Yue Wu
  • Benjamin Grimmer
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    This work considers the nonconvex, nonsmooth problem of minimizing a composite objective of the form $f(g(x))+h(x)$ where the inner mapping $g$ is a smooth finite summation or expectation amenable to variance reduction. In such settings, prox-linear methods can enjoy variance-reduced speed-ups despite the existence of nonsmoothness. We provide a unified convergence theory applicable to a … Read more