Nicolas Couellan – Optimization Online

On the convergence of stochastic bi-level gradient methods

Published: 2016/02/15

Data-Mining, Nonlinear Optimization bi-level optimization, empirical risk minimization, gradient approximation, stochastic gradient

We analyze the convergence of stochastic gradient methods for bi-level optimization problems. We address two specific cases: first when the outer objective function can be expressed as a finite sum of independent terms, and next when both the outer and inner objective functions can be expressed as finite sums of independent terms. We assume Lipschitz … Read more

Incremental Accelerated Gradient Methods for SVM Classification: Study of the Constrained Approach

Published: 2013/05/02, Updated: 2013/08/30

Nicolas Couellan

Sophie Jan

Constrained Nonlinear Optimization, Data-Mining accelerated gradient, constrained gradient, incremental gradient method, kernel technique, machine learning, nonlinear programming, support vector machines

We investigate constrained first order techniques for training Support Vector Machines (SVM) for online classification tasks. The methods exploit the structure of the SVM training problem and combine ideas of incremental gradient technique, gradient acceleration and successive simple calculations of Lagrange multipliers. Both primal and dual formulations are studied and compared. Experiments show that the … Read more