Convex Optimization

Unifying restart accelerated gradient and proximal bundle methods

  • Jiaming Liang
    • Categories Convex and Nonsmooth Optimization, Convex Optimization Tags , , , ,

    This paper presents a novel restarted version of Nesterov’s accelerated gradient method and establishes its optimal iteration-complexity for solving convex smooth composite optimization problems. The proposed restart accelerated gradient method is shown to be a specific instance of the accelerated inexact proximal point framework introduced in “An accelerated hybrid proximal extragradient method for convex optimization … Read more

    Primal-dual proximal bundle and conditional gradient methods for convex problems

  • Jiaming Liang
    • Categories Convex and Nonsmooth Optimization, Convex Optimization, Nonsmooth Optimization Tags , , , , ,

    This paper studies the primal-dual convergence and iteration-complexity of proximal bundle methods for solving nonsmooth problems with convex structures. More specifically, we develop a family of primal-dual proximal bundle methods for solving convex nonsmooth composite optimization problems and establish the iteration-complexity in terms of a primal-dual gap. We also propose a class of proximal bundle … Read more

    A subgradient splitting algorithm for optimization on nonpositively curved metric spaces

  • Ariel Goodwin
  • Adrian Lewis
    • Categories Convex Optimization, Generalized Convexity/Monoticity, Nonsmooth Optimization

    Many of the primal ingredients of convex optimization extend naturally from Euclidean to Hadamard spaces — nonpositively curved metric spaces like Euclidean, Hilbert, and hyperbolic spaces, metric trees, and more general CAT(0) cubical complexes. Linear structure, however, and the duality theory it supports are absent. Nonetheless, we introduce a new type of subgradient for convex … Read more

    Inexact FISTA-like Methods with Adaptive Backtracking

  • Gabriel Grillo
  • Sandra A. Santos
  • Elias Salomão Helou
    • Categories Biomedical Applications, Convex and Nonsmooth Optimization, Convex Optimization Tags , ,

    Accelerated proximal gradient methods have become a useful tool in large-scale convex optimization, specially for variational regularization with non-smooth priors. Prevailing convergence analysis considers that users can perform the proximal and the gradient steps exactly. Still, in some practical applications, the proximal or the gradient steps must be computed inexactly, which can harm convergence speed … Read more

    Addressing Estimation Errors through Robust Portfolio Optimization

  • Gérard Cornuéjols
  • Ozgun Elci
  • Vrishabh Patil
    • Categories Convex Optimization, Quadratic Programming, Robust Optimization Tags , ,

    It is well known that the performance of the classical Markowitz model for portfolio optimization is extremely sensitive to estimation errors on the expected asset returns. Robust optimization mitigates this issue. We focus on ellipsoidal uncertainty sets around a point estimate of the expected asset returns. An important issue is the choice of the parameters … Read more

    An inertial Riemannian gradient ADMM for nonsmooth manifold optimization

  • Xiaoquan Wang
    • Categories Convex Optimization, Nonsmooth Optimization, Optimization of Systems modeled by PDEs

    The Alternating Direction Method of Multipliers (ADMM) is widely recognized for its efficiency in solving separable optimization problems. However, its application to optimization on Riemannian manifolds remains a significant challenge. In this paper, we propose a novel inertial Riemannian gradient ADMM (iRG-ADMM) to solve Riemannian optimization problems with nonlinear constraints. Our key contributions are as … Read more

    Accelerating Proximal Gradient Descent via Silver Stepsizes

  • Jinho Bok
  • Jason M. Altschuler
    • Categories Convex Optimization

    Surprisingly, recent work has shown that gradient descent can be accelerated without using momentum–just by judiciously choosing stepsizes. An open question raised by several papers is whether this phenomenon of stepsize-based acceleration holds more generally for constrained and/or composite convex optimization via projected and/or proximal versions of gradient descent. We answer this in the affirmative … Read more

    Acceleration by Random Stepsizes: Hedging, Equalization, and the Arcsine Stepsize Schedule

  • Jason M. Altschuler
  • Pablo A. Parrilo
    • Categories Convex Optimization

    We show that for separable convex optimization, random stepsizes fully accelerate Gradient Descent. Specifically, using inverse stepsizes i.i.d. from the Arcsine distribution improves the iteration complexity from $O(k)$ to $O(k^{1/2})$, where $k$ is the condition number. No momentum or other algorithmic modifications are required. This result is incomparable to the (deterministic) Silver Stepsize Schedule which … Read more

    Reduced Sample Complexity in Scenario-Based Control System Design via Constraint Scaling

  • Jaeseok Choi
  • Constantino Lagoa
  • Anirudh Subramanyam
    • Categories Control Applications, Convex Optimization, Stochastic Programming Tags , ,

    The scenario approach is widely used in robust control system design and chance-constrained optimization, maintaining convexity without requiring assumptions about the probability distribution of uncertain parameters. However, the approach can demand large sample sizes, making it intractable for safety-critical applications that require very low levels of constraint violation. To address this challenge, we propose a … Read more

    Jordan and isometric cone automorphisms in Euclidean Jordan algebras

  • Michael Orlitzky
    • Categories Cone Programming, Convex Optimization, Linear, Cone and Semidefinite Programming Tags , , ,

    Every symmetric cone K arises as the cone of squares in a Euclidean Jordan algebra V. As V is a real inner-product space, we may denote by Isom(V) its group of isometries. The groups JAut(V) of its Jordan-algebra automorphisms and Aut(K) of the linear cone automorphisms are then related. For certain inner products, JAut(V) = … Read more