rate of convergence – Page 2 – Optimization Online

Convergence Rate of Stochastic Gradient Search in the Case of Multiple and Non-Isolated Minima

Published: 2009/04/27

Control Applications, Statistics, Stochastic Programming arma models, lojasiewicz inequalities, machine learning, rate of convergence, recursive prediction error, stochastic gradient algorithms, supervised learning, system identification, temporal-difference learning

The convergence rate of stochastic gradient search is analyzed in this paper. Using arguments based on differential geometry and Lojasiewicz inequalities, tight bounds on the convergence rate of general stochastic gradient algorithms are derived. As opposed to the existing results, the results presented in this paper allow the objective function to have multiple, non-isolated minima, … Read more

The Rate of Convergence of the Augmented Lagrangian Method for Nonlinear Semidefinite Programming

Published: 2006/01/25

Jie Sun

Defeng Sun

Liwei Zhang

Linear, Cone and Semidefinite Programming, Nonlinear Optimization augmented lagrangian method, nonlinear semidefinite programming, rate of convergence, variational analysis

We analyze the rate of local convergence of the augmented Lagrangian method for nonlinear semidefinite optimization. The presence of the positive semidefinite cone constraint requires extensive tools such as the singular value decomposition of matrices, an implicit function theorem for semismooth functions, and certain variational analysis on the projection operator in the symmetric-matrix space. Without … Read more