parameter-free – Optimization Online

Optimal Nonergodic Primal-Dual Complexity of Efficient Inexact Parameter-Free Augmented Lagrangian Methods

Published: 2026/08/04, Updated: 2026/08/10

Augmented Lagrangian (AL) methods are a classical framework for constrained optimization, but for directly verifiable approximate KKT points, known first-order complexity bounds for standard inexact AL methods are suboptimal, while the best known proximal augmented Lagrangian (PAL) bounds retain an additional logarithmic factor. We consider linearly constrained convex composite problems with a smooth convex term … Read more

A Simple Adaptive Proximal Gradient Method for Nonconvex Optimization

Published: 2025/10/08

Constrained Nonlinear Optimization, Nonsmooth Optimization adaptive proximal gradient, composite optimization, Line-Search-Free, Lipschitz gradient, parameter-free

Consider composite nonconvex optimization problems where the objective function consists of a smooth nonconvex term (with Lipschitz-continuous gradient) and a convex (possibly nonsmooth) term. Existing parameter-free methods for such problems often rely on complex multi-loop structures, require line searches, or depend on restrictive assumptions (e.g., bounded iterates). To address these limitations, we introduce a novel … Read more

An Adaptive and Parameter-Free Nesterov’s Accelerated Gradient Method for Convex Optimization

Published: 2025/05/16

Convex Optimization Adaptive methods, convex optimization, first-order methods, Line-Search-Free, parameter-free

We propose AdaNAG, an adaptive accelerated gradient method based on Nesterov’s accelerated gradient method. AdaNAG is line-search-free, parameter-free, and achieves the accelerated convergence rates \( f(x_k) – f_\star = \mathcal{O}\left(1/k^2\right) \) and \( \min_{i\in\left\{1,\dots, k\right\}} \|\nabla f(x_i)\|^2 = \mathcal{O}\left(1/k^3\right) \) for \( L \)-smooth convex function \( f \). We provide a Lyapunov analysis for … Read more

AdaBB: Adaptive Barzilai-Borwein Method for Convex Optimization

Published: 2024/01/15

Convex Optimization Adaptive Gradient Descent, Automated Gradient Descent, barzilai-borwein stepsize, Line-Search-Free, Locally Lipschitz Gradient, parameter-free

In this paper, we propose AdaBB, an adaptive gradient method based on the Barzilai-Borwein stepsize. The algorithm is line-search-free and parameter-free, and essentially provides a convergent variant of the Barzilai-Borwein method for general unconstrained convex optimization. We analyze the ergodic convergence of the objective function value and the convergence of the iterates for solving general … Read more

A nearly linearly convergent first-order method for nonsmooth functions with quadratic growth

Published: 2022/04/29, Updated: 2022/05/04

Convex and Nonsmooth Optimization active manifold, decomposable, Goldstein, Kurdyka-Lojasiewicz, max-of-smooth, nonsmooth, parameter-free, quadratic growth, semialgebraic, subgradient method

Classical results show that gradient descent converges linearly to minimizers of smooth strongly convex functions. A natural question is whether there exists a locally nearly linearly convergent method for nonsmooth functions with quadratic growth. This work designs such a method for a wide class of nonsmooth and nonconvex locally Lipschitz functions, including max-of-smooth, Shapiro’s decomposable … Read more