svrg

Acceleration of SVRG and Katyusha X by Inexact Preconditioning

  • Fei Feng
  • Yanli Liu
  • Wotao Yin
    • Categories Convex Optimization, Stochastic Programming Tags , ,

    Empirical risk minimization is an important class of optimization problems with many popular machine learning applications, and stochastic variance reduction methods are popular choices for solving them. Among these methods, SVRG and Katyusha X (a Nesterov accelerated SVRG) achieve fast convergence without substantial memory requirement. In this paper, we propose to accelerate these two algorithms … Read more

    A SMART Stochastic Algorithm for Nonconvex Optimization with Applications to Robust Machine Learning

  • Aleksandr Y. Aravkin
  • Damek Davis
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Statistics Tags , , , , , ,

    Machine learning theory typically assumes that training data is unbiased and not adversarially generated. When real training data deviates from these assumptions, trained models make erroneous predictions, sometimes with disastrous effects. Robust losses, such as the huber norm are designed to mitigate the effects of such contaminated data, but they are limited to the regression … Read more