parameter-free – Optimization Online

AdaBB: Adaptive Barzilai-Borwein Method for Convex Optimization

Published: 2024/01/15

Convex Optimization Adaptive Gradient Descent, Automated Gradient Descent, barzilai-borwein stepsize, Line-Search-Free, Locally Lipschitz Gradient, parameter-free

In this paper, we propose AdaBB, an adaptive gradient method based on the Barzilai-Borwein stepsize. The algorithm is line-search-free and parameter-free, and essentially provides a convergent variant of the Barzilai-Borwein method for general unconstrained convex optimization. We analyze the ergodic convergence of the objective function value and the convergence of the iterates for solving general … Read more

A nearly linearly convergent first-order method for nonsmooth functions with quadratic growth

Published: 2022/04/29, Updated: 2022/05/04

Damek Davis

Liwei Jiang

Convex and Nonsmooth Optimization active manifold, decomposable, Goldstein, Kurdyka-Lojasiewicz, max-of-smooth, nonsmooth, parameter-free, quadratic growth, semialgebraic, subgradient method

Classical results show that gradient descent converges linearly to minimizers of smooth strongly convex functions. A natural question is whether there exists a locally nearly linearly convergent method for nonsmooth functions with quadratic growth. This work designs such a method for a wide class of nonsmooth and nonconvex locally Lipschitz functions, including max-of-smooth, Shapiro’s decomposable … Read more