barzilai-borwein stepsize – Optimization Online

AdaBB: Adaptive Barzilai-Borwein Method for Convex Optimization

Published: 2024/01/15

Convex Optimization Adaptive Gradient Descent, Automated Gradient Descent, barzilai-borwein stepsize, Line-Search-Free, Locally Lipschitz Gradient, parameter-free

In this paper, we propose AdaBB, an adaptive gradient method based on the Barzilai-Borwein stepsize. The algorithm is line-search-free and parameter-free, and essentially provides a convergent variant of the Barzilai-Borwein method for general unconstrained convex optimization. We analyze the ergodic convergence of the objective function value and the convergence of the iterates for solving general … Read more

A Non-monotonic Method for Large-scale Nonnegative Least Squares

Published: 2010/05/26

Inderjit S. Dhillon

Dongmin Kim

Suvrit Sra

Basic Sciences Applications, Bound-constrained Optimization, Nonlinear Systems and Least-Squares barzilai-borwein stepsize, gradient projection methods, large-scale optimization, least squares, nnls, non-monotonic descent, nonnegativity constraints

We present a new algorithm for nonnegative least-squares (NNLS). Our algorithm extends the unconstrained quadratic optimization algorithm of Barzilai and Borwein (BB) (J. Barzilai and J. M. Borwein; Two-Point Step Size Gradient Methods. IMA J. Numerical Analysis; 1988.) to handle nonnegativity constraints. Our extension differs in several basic aspects from other constrained BB variants. The … Read more