Convex Optimization

Optimal Nonergodic Primal-Dual Complexity of Efficient Inexact Parameter-Free Augmented Lagrangian Methods

Categories Constrained Nonlinear Optimization, Convex Optimization, Nonlinear Optimization Tags , , , ,

We develop inexact augmented Lagrangian (AL) methods for linearly constrained convex composite optimization problems whose objective is the sum of a smooth convex function and a possibly nonsmooth closed proper convex function with compact domain. Unlike primal accuracy guarantees based on objective-value error, our methods target verifiable approximate KKT solutions. In the convex setting, we … Read more

Entropy-Smooth Convex Optimization Cannot Be Accelerated

Categories Convex and Nonsmooth Optimization, Convex Optimization, Optimization in Data Science Tags , ,

We prove an $\Omega(L/T)$ lower bound for the convergence rate of minimization in the class of functions that are convex and $L$-smooth relative to negative entropy on the standard $d$-simplex, valid for every first-order method when $d = \Omega(T^2)$. In particular, this shows that mirror descent is optimal up to a logarithmic factor in this … Read more

On the boundedness of infinite products of relaxed projections: perturbations resilience and dynamic string-averaging

Categories Basic Sciences Applications, Convex Optimization, Infinite Dimensional Optimization Tags , , , , , , , , ,

Very recently (2026), Bauschke and Tung extended from finite- to infinite-dimensional Hilbert spaces a result published by Meshulam in 1996 (following an earlier result of Aharoni-Duchet-Wajnryb from 1984) regarding the boundedness of infinite products of relaxed projections onto a finite family of closed affine subspaces. In the present note we extend in various ways the … Read more

Automorphisms of hyperbolic polynomials

Categories Convex Optimization, Linear, Cone and Semidefinite Programming Tags , , , ,

The pair \( (p,e) \) is hyperbolic if \( p : \mathbb{R}^{n} \to \mathbb{R} \) is a homogeneous polynomial, if \( e \in \mathbb{R}^{n} \), if \( p(e) > 0 \), and if the roots of \( t \mapsto p(te – x) \) are real for all \( x \in \mathbb{R}^{n} \). In that case, … Read more

Inexactly Smooth Performance Estimation and New Optimized Gradient Methods

Categories Convex Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

  We consider a general class of “inexactly smooth” convex functions, providing a universal model capturing as special cases $L$-smooth, $M$-Lipschitz, and H\”older smooth functions, and any combination thereof. Such functions possess a calculus closely following that of smooth functions. Our main results provide inexactly smooth functions with interpolation theorems that are necessary and sufficient … Read more

A Parameter-Free Restart Scheme with Only a Parallelizable $\log\log(1/\epsilon)$ Overhead

Categories Convex Optimization, Nonsmooth Optimization Tags , , ,

It is well-known that first-order methods can offer accelerated convergence rates in the presence of growth structures. Restarting schemes provide a general tool for such speed-ups. These schemes typically either require unrealistic problem knowledge, incur logarithmic overhead factors in oracle complexity, and/or have a nontrivial initial burn-in phase. We present a parameter-free approach for restarting … Read more

Disjunctive Sum of Squares

Categories Cone Programming, Convex Optimization, Semi-definite Programming Tags , , , , , , ,

We introduce the concept of disjunctive sum of squares for certifying nonnegativity of polynomials. Unlike the popular sum of squares approach where nonnegativity is certified by a single algebraic identity, the disjunctive sum of squares approach certifies nonnegativity with multiple algebraic identities which can be found in parallel. Our main result is a disjunctive Positivstellensatz … Read more

Log-Averaged Mirror Prox for Fast, Large-Scale Optimal Transport in Linear Space

Categories Convex Optimization, Parallel Algorithms Tags , , , ,

We propose Log-Averaged Mirror Prox (LAMP), a linear-space primal-dual method for large-scale optimal transport. LAMP implements primal mirror prox updates by tracking an averaged dual sequence, reducing storage complexity from \({O}(nm)\) to \({O}(n+m)\) while preserving dense, GPU-friendly reductions. Consequently, LAMP preserves the last-iterate \(\widetilde{{O}}( nm\varepsilon^{-1})\) arithmetic complexity of conservatively parameterized primal-dual mirror prox. We further … Read more

Inertial forward-backward methods with subgradient-based corrections

Categories Convex and Nonsmooth Optimization, Convex Optimization, Nonsmooth Optimization Tags , , ,

Shi et al. \cite{shi2022understanding} propose acceleration methods to solve smooth convex optimization problems. In our work, we focus on the general unconstrained composite non-smooth convex optimization problem. We provide an inertial forward-backward algorithm with subgradient correction, derived through time discretization of the ODE, as studied by Shi et al. We achieve the rate of convergence … Read more

Stochastic Gradient Methods with Online Scaling

Categories Convex Optimization, Stochastic Programming

This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in \cite{gao2025gradient,chu2025gradient} to stochastic optimization. Under standard assumptions, we establish convergence guarantees for SOSGM using large batchsize or variance reduction. SOSGM is compatible with popular diagonal and/or low-rank preconditioners as well as heavy-ball momentum, while maintaining memory … Read more