stochastic newton methods – Optimization Online

A Fully Stochastic Second-Order Trust Region Method

Published: 2019/11/15

Nonlinear Optimization, Stochastic Programming, Unconstrained Optimization deep neural networks, finite-sum optimization, machine learning, stochastic newton methods, stochastic optimization, time series forecasting, trust-region methods

A stochastic second-order trust region method is proposed, which can be viewed as a second-order extension of the trust-region-ish (TRish) algorithm proposed by Curtis et al. [INFORMS J. Optim. 1(3) 200–220, 2019]. In each iteration, a search direction is computed by (approximately) solving a trust region subproblem defined by stochastic gradient and Hessian estimates. The … Read more