Convex Optimization

Optimal Nonergodic Primal-Dual Complexity of Efficient Inexact Parameter-Free Augmented Lagrangian Methods

Categories Constrained Nonlinear Optimization, Convex Optimization, Nonlinear Optimization Tags , , , ,

Augmented Lagrangian (AL) methods are a classical framework for constrained optimization, but for directly verifiable approximate KKT points, known first-order complexity bounds for standard inexact AL methods are suboptimal, while the best known proximal augmented Lagrangian (PAL) bounds retain an additional logarithmic factor. We consider linearly constrained convex composite problems with a smooth convex term … Read more

Entropy-Smooth Convex Optimization Cannot Be Accelerated

Categories Convex and Nonsmooth Optimization, Convex Optimization, Optimization in Data Science Tags , ,

We prove an $\Omega(L/T)$ lower bound for the convergence rate of minimization in the class of functions that are convex and $L$-smooth relative to negative entropy on the standard $d$-simplex, valid for every first-order method when $d = \Omega(T^2)$. In particular, this shows that mirror descent is optimal up to a logarithmic factor in this … Read more

On the boundedness of infinite products of relaxed projections: perturbations resilience and dynamic string-averaging

Categories Basic Sciences Applications, Convex Optimization, Infinite Dimensional Optimization Tags , , , , , , , , ,

Very recently (2026), Bauschke and Tung extended from finite- to infinite-dimensional Hilbert spaces a result published by Meshulam in 1996 (following an earlier result of Aharoni-Duchet-Wajnryb from 1984) regarding the boundedness of infinite products of relaxed projections onto a finite family of closed affine subspaces. In the present note we extend in various ways the … Read more

Automorphisms of hyperbolic polynomials

Categories Convex Optimization, Linear, Cone and Semidefinite Programming Tags , , , ,

The pair \( (p,e) \) is hyperbolic if \( p : \mathbb{R}^{n} \to \mathbb{R} \) is a homogeneous polynomial, if \( e \in \mathbb{R}^{n} \), if \( p(e) > 0 \), and if the roots of \( t \mapsto p(te – x) \) are real for all \( x \in \mathbb{R}^{n} \). In that case, … Read more

Inexactly Smooth Performance Estimation and New Optimized Gradient Methods

Categories Convex Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

  We consider a general class of “inexactly smooth” convex functions, providing a universal model capturing as special cases $L$-smooth, $M$-Lipschitz, and H\”older smooth functions, and any combination thereof. Such functions possess a calculus closely following that of smooth functions. Our main results provide inexactly smooth functions with interpolation theorems that are necessary and sufficient … Read more

A Parameter-Free Restart Scheme with Only a Parallelizable $\log\log(1/\epsilon)$ Overhead

Categories Convex Optimization, Nonsmooth Optimization Tags , , ,

It is well-known that first-order methods can offer accelerated convergence rates in the presence of growth structures. Restarting schemes provide a general tool for such speed-ups. These schemes typically either require unrealistic problem knowledge, incur logarithmic overhead factors in oracle complexity, and/or have a nontrivial initial burn-in phase. We present a parameter-free approach for restarting … Read more

Disjunctive Sum of Squares

Categories Cone Programming, Convex Optimization, Semi-definite Programming Tags , , , , , , ,

We introduce the concept of disjunctive sum of squares for certifying nonnegativity of polynomials. Unlike the popular sum of squares approach where nonnegativity is certified by a single algebraic identity, the disjunctive sum of squares approach certifies nonnegativity with multiple algebraic identities which can be found in parallel. Our main result is a disjunctive Positivstellensatz … Read more

Log-Averaged Mirror Prox for Fast, Large-Scale Optimal Transport in Linear Space

Categories Convex Optimization, Parallel Algorithms Tags , , , ,

We propose Log-Averaged Mirror Prox (LAMP), a linear-space primal-dual method for large-scale optimal transport. LAMP implements primal mirror prox updates by tracking an averaged dual sequence, reducing storage complexity from \({O}(nm)\) to \({O}(n+m)\) while preserving dense, GPU-friendly reductions. Consequently, LAMP preserves the last-iterate \(\widetilde{{O}}( nm\varepsilon^{-1})\) arithmetic complexity of conservatively parameterized primal-dual mirror prox. We further … Read more

Inertial forward-backward methods with subgradient-based corrections

Categories Convex and Nonsmooth Optimization, Convex Optimization, Nonsmooth Optimization Tags , , ,

Shi et al. \cite{shi2022understanding} propose acceleration methods to solve smooth convex optimization problems. In our work, we focus on the general unconstrained composite non-smooth convex optimization problem. We provide an inertial forward-backward algorithm with subgradient correction, derived through time discretization of the ODE, as studied by Shi et al. We achieve the rate of convergence … Read more

Stochastic Gradient Methods with Online Scaling

Categories Convex Optimization, Stochastic Programming

This paper introduces Stochastic Online Scaled Gradient Methods (SOSGM), a generalization of the recently developed adaptive preconditioning framework in \cite{gao2025gradient,chu2025gradient} to stochastic optimization. Under standard assumptions, we establish convergence guarantees for SOSGM using large batchsize or variance reduction. SOSGM is compatible with popular diagonal and/or low-rank preconditioners as well as heavy-ball momentum, while maintaining memory … Read more