In numerical optimization, line-search and trust-region methods are two important classes of descent schemes, with well-understood global convergence properties. Here we consider “accelerated” versions of these methods, where the conventional iterate is allowed to be replaced by any point that produces at least as much decrease in the cost function as a fixed fraction of the decrease produced by the conventional iterate. A detailed convergence analysis reveals that global convergence properties of line-search and trust-region methods still hold when the methods are accelerated. The analysis is performed in the general context of optimization on manifolds, of which optimization in R^n is a particular case. This general convergence analysis sheds a new light on the behavior of several existing algorithms.
Citation
http://dx.doi.org/10.1137/08072019X : SIAM Journal on Numerical Analysis, Vol. 47, No. 2, pp. 997-1018, 2009