incremental gradient method

A Proximal Stochastic Gradient Method with Progressive Variance Reduction

  • Lin Xiao
  • Tong Zhang
    • Categories Convex and Nonsmooth Optimization, Stochastic Programming Tags , ,

    We consider the problem of minimizing the sum of two convex functions: one is the average of a large number of smooth component functions, and the other is a general convex function that admits a simple proximal mapping. We assume the whole objective function is strongly convex. Such problems often arise in machine learning, known … Read more

    Incremental Accelerated Gradient Methods for SVM Classification: Study of the Constrained Approach

  • Nicolas Couellan
  • Sophie Jan
    • Categories Constrained Nonlinear Optimization, Data-Mining Tags , , , , , ,

    We investigate constrained first order techniques for training Support Vector Machines (SVM) for online classification tasks. The methods exploit the structure of the SVM training problem and combine ideas of incremental gradient technique, gradient acceleration and successive simple calculations of Lagrange multipliers. Both primal and dual formulations are studied and compared. Experiments show that the … Read more