finite sum minimization – Optimization Online

SLiSeS: Subsampled Line Search Spectral Gradient Method for Finite Sums

Published: 2023/06/01, Updated: 2024/02/19

Data Science Algorithms, Nonlinear Optimization, Optimization in Data Science finite sum minimization, line-search, spectral gradient methods, subsampling

Citation SLiSes Article Download View SLiSeS: Subsampled Line Search Spectral Gradient Method for Finite Sums

A stochastic first-order trust-region method with inexact restoration for finite-sum minimization

Published: 2021/06/22

Nonlinear Optimization finite sum minimization, inexact restoration, sub-sampling, trust-region methods, worst-case evaluation complexity

We propose a stochastic first-order trust-region method with inexact function and gradient evaluations for solving finite-sum minimization problems. At each iteration, the function and the gradient are approximated by sampling. The sample size in gradient approximations is smaller than the sample size in function approximations and the latter is determined using a deterministic rule inspired … Read more

A Stochastic Trust Region Algorithm Based on Careful Step Normalization

Published: 2017/12/29, Updated: 2018/06/26

Frank E. Curtis

Katya Scheinberg

Rui Shi

Nonlinear Optimization, Stochastic Programming, Unconstrained Optimization deep neural networks, finite sum minimization, logistic regression, machine learning, stochastic gradient method, stochastic optimization, trust-region methods

An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm—which dynamically chooses whether or not to employ normalized steps—is proved to have … Read more