second-order approximation – Optimization Online

Inexact proximal stochastic second-order methods for nonconvex composite optimization

Published: 2019/10/10, Updated: 2019/10/14

Nonlinear Optimization (weakly) smooth function, complexity, inexact subproblem solution, nonconvex, second-order approximation, stochastic gradient, variance reduction

In this paper, we propose a framework of Inexact Proximal Stochastic Second-order (IPSS) methods for solving nonconvex optimization problems, whose objective function consists of an average of finitely many, possibly weakly, smooth functions and a convex but possibly nons- mooth function. At each iteration, IPSS inexactly solves a proximal subproblem constructed by using some positive … Read more

Inexact Successive Quadratic Approximation for Regularized Optimization

Published: 2018/03/03, Updated: 2019/01/24

Stephen Wright

Ching-pei Lee

Nonlinear Optimization, Nonsmooth Optimization convex optimization, inexact method, nonconvex optimization, proximal method, regularized optimization, second-order approximation, variable metric

Successive quadratic approximations, or second-order proximal methods, are useful for minimizing functions that are a sum of a smooth part and a convex, possibly nonsmooth part that promotes regularization. Most analyses of iteration complexity focus on the special case of proximal gradient method, or accelerated variants thereof. There have been only a few studies of … Read more