Convex and Nonsmooth Optimization

How Stringent is the Linear Independence Kink Qualification in Abs-Smooth Optimization?

  • Franz Bethke
  • Andrea Walther
    • Categories Nonsmooth Optimization Tags

    Abs-smooth functions are given by compositions of smooth functions and the evaluation of the absolute value. The linear independence kink qualification (LIKQ) is a fundamental assumption in optimization problems governed by these abs-smooth functions, generalizing the well-known LICQ from smooth optimization. In particular, provided that LIKQ holds it is possible to derive optimality conditions for … Read more

    A proximal alternating direction method of multipliers with a proximal-perturbed Lagrangian function for nonconvex and nonsmooth structured optimization

  • Pengjie Liu
    • Categories Nonsmooth Optimization Tags , , ,

    Building on [J. Glob. Optim. 89 (2024) 899–926], we continue to focus on solving a nonconvex and nonsmooth structured optimization problem with linear and closed convex set constraints, where its objective function is the sum of a convex (possibly nonsmooth) function and a smooth (possibly nonconvex) function. Based on the traditional augmented Lagrangian construction, we … Read more

    Accelerating Proximal Gradient Descent via Silver Stepsizes

  • Jinho Bok
  • Jason M. Altschuler
    • Categories Convex Optimization

    Surprisingly, recent work has shown that gradient descent can be accelerated without using momentum–just by judiciously choosing stepsizes. An open question raised by several papers is whether this phenomenon of stepsize-based acceleration holds more generally for constrained and/or composite convex optimization via projected and/or proximal versions of gradient descent. We answer this in the affirmative … Read more

    Acceleration by Random Stepsizes: Hedging, Equalization, and the Arcsine Stepsize Schedule

  • Jason M. Altschuler
  • Pablo A. Parrilo
    • Categories Convex Optimization

    We show that for separable convex optimization, random stepsizes fully accelerate Gradient Descent. Specifically, using inverse stepsizes i.i.d. from the Arcsine distribution improves the iteration complexity from $O(k)$ to $O(k^{1/2})$, where $k$ is the condition number. No momentum or other algorithmic modifications are required. This result is incomparable to the (deterministic) Silver Stepsize Schedule which … Read more

    A proximal-perturbed Bregman ADMM for solving nonsmooth and nonconvex optimization problems

  • Jianchao Bai
    • Categories Nonsmooth Optimization Tags , , ,

    In this paper, we focus on a linearly constrained composite minimization problem whose objective function is possibly nonsmooth and nonconvex. Unlike the traditional construction of augmented Lagrangian function, we provide a proximal-perturbed augmented Lagrangian and then develop a new Bregman Alternating Direction Method of Multipliers (ADMM). Under mild assumptions, we show that the novel augmented … Read more

    Reduced Sample Complexity in Scenario-Based Control System Design via Constraint Scaling

  • Jaeseok Choi
  • Constantino Lagoa
  • Anirudh Subramanyam
    • Categories Control Applications, Convex Optimization, Stochastic Programming Tags , ,

    The scenario approach is widely used in robust control system design and chance-constrained optimization, maintaining convexity without requiring assumptions about the probability distribution of uncertain parameters. However, the approach can demand large sample sizes, making it intractable for safety-critical applications that require very low levels of constraint violation. To address this challenge, we propose a … Read more

    The Proximal Bundle Algorithm Under a Frank-Wolfe Perspective: an Improved Complexity Analysis

  • David Fersztand
  • Xu Andy Sun
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , ,

    The proximal bundle algorithm (PBA) is a fundamental and computationally effective algorithm for solving optimization problems with nonsmooth components. We investigate its convergence rate, focusing on composite settings where one function is smooth and the other is piecewise linear. We interpret a sequence of null steps of the PBA as a Frank-Wolfe algorithm on the … Read more

    Jordan and isometric cone automorphisms in Euclidean Jordan algebras

  • Michael Orlitzky
    • Categories Cone Programming, Convex Optimization, Linear, Cone and Semidefinite Programming Tags , , ,

    Every symmetric cone K arises as the cone of squares in a Euclidean Jordan algebra V. As V is a real inner-product space, we may denote by Isom(V) its group of isometries. The groups JAut(V) of its Jordan-algebra automorphisms and Aut(K) of the linear cone automorphisms are then related. For certain inner products, JAut(V) = … Read more

    New efficient accelerated and stochastic gradient descent algorithms based on locally Lipschitz gradient constants

  • Pham Thi Hoai
    • Categories Convex Optimization, Nonlinear Optimization, Stochastic Programming Tags , , , , , , ,

    In this paper, we revisit the recent stepsize applied for the gradient descent scheme which is called NGD proposed by [Hoai et al., A novel stepsize for gradient descent method, Operations Research Letters (2024) 53, doi: \href{10.1016/j.orl.2024.107072}{10.1016/j.orl.2024.107072}]. We first investigate NGD stepsize with two well-known accelerated techniques which are Heavy ball and Nesterov’s methods. In … Read more

    Gradient Methods with Online Scaling

  • Wenzhi Gao
    • Categories Convex Optimization, Nonlinear Optimization, Other Topics Tags , ,

    We introduce a framework to accelerate the convergence of gradient-based methods with online learning. The framework learns to scale the gradient at each iteration through an online learning algorithm and provably accelerates gradient-based methods asymptotically. In contrast with previous literature, where convergence is established based on worst-case analysis, our framework provides a strong convergence guarantee … Read more