Convex Optimization

Proximal Alternating Penalty Algorithms for Nonsmooth Constrained Convex Optimization

  • Quoc Tran-Dinh
    • Categories Convex and Nonsmooth Optimization, Convex Optimization

    We develop two new proximal alternating penalty algorithms to solve a wide range class of constrained convex optimization problems. Our approach mainly relies on a novel combination of the classical quadratic penalty, alternating, Nesterov’s acceleration, and homotopy techniques. The first algorithm is designed to solve generic and possibly nonsmooth constrained convex problems without requiring any … Read more

    Douglas-Rachford Splitting for Pathological Convex Optimization

  • Yanli Liu
  • Ernest K. Ryu
  • Wotao Yin
    • Categories Convex Optimization Tags , , , , , , ,

    Despite the vast literature on DRS, there has been very little work analyzing their behavior under pathologies. Most analyses assume a primal solution exists, a dual solution exists, and strong duality holds. When these assumptions are not met, i.e., under pathologies, the theory often breaks down and the empirical performance may degrade significantly. In this … Read more

    Non-stationary Douglas-Rachford and alternating direction method of multipliers: adaptive stepsizes and convergence

  • Dirk A. Lorenz
  • Quoc Tran-Dinh
    • Categories Convex Optimization Tags , , , ,

    We revisit the classical Douglas-Rachford (DR) method for finding a zero of the sum of two maximal monotone operators. Since the practical performance of the DR method crucially depends on the stepsizes, we aim at developing an adaptive stepsize rule. To that end, we take a closer look at a linear case of the problem … Read more

    Simplified Versions of the Conditional Gradient Method

  • Igor V. Konnov
    • Categories Constrained Nonlinear Optimization, Convex Optimization

    We suggest simple modifications of the conditional gradient method for smooth optimization problems, which maintain the basic convergence properties, but reduce the implementation cost of each iteration essentially. Namely, we propose the step-size procedure without any line-search, and inexact solution of the direction finding subproblem. Preliminary results of computational tests confirm efficiency of the proposed … Read more

    A forward-backward penalty scheme with inertial effects for montone inclusions. Applications to convex bilevel programming

  • Radu Ioan Bot
  • Dang-Khoa Nguyen
    • Categories Convex Optimization Tags , , ,

    We investigate forward-backward splitting algorithm of penalty type with inertial effects for finding the zeros of the sum of a maximally monotone operator and a cocoercive one and the convex normal cone to the set of zeroes of an another cocoercive operator. Weak ergodic convergence is obtained for the iterates, provided that a condition express … Read more

    Convergence rates of proximal gradient methods via the convex conjugate

  • David H. Gutman
  • Javier F. Peña
    • Categories Convex Optimization Tags , ,

    We give a novel proof of the $O(1/k)$ and $O(1/k^2)$ convergence rates of the proximal gradient and accelerated proximal gradient methods for composite convex minimization. The crux of the new proof is an upper bound constructed via the convex conjugate of the objective function. CitationTechnical Report, Carnegie Mellon University, January 2018.ArticleDownload View PDF

    A Random Block-Coordinate Douglas-Rachford Splitting Method with Low Computational Complexity for Binary Logistic Regression

  • Luis M. Briceno-Arias
  • Émilie Chouzenoux
  • Jean-Christophe Pesquet
  • Giovanni Chierchia
    • Categories Convex Optimization, Statistics Tags , , ,

    In this paper, we propose a new optimization algorithm for sparse logistic regression based on a stochastic version of the Douglas Rachford splitting method. Our algorithm sweeps the training set by randomly selecting a mini-batch of data at each iteration, and it allows us to update the variables in a block coordinate manner. Our approach … Read more

    A single potential governing convergence of conjugate gradient, accelerated gradient and geometric descent

  • Sahar Karimi
  • Stephen A. Vavasis
    • Categories Convex Optimization Tags , , ,

    Nesterov’s accelerated gradient (AG) method for minimizing a smooth strongly convex function $f$ is known to reduce $f({\bf x}_k)-f({\bf x}^*)$ by a factor of $\epsilon\in(0,1)$ after $k=O(\sqrt{L/\ell}\log(1/\epsilon))$ iterations, where $\ell,L$ are the two parameters of smooth strong convexity. Furthermore, it is known that this is the best possible complexity in the function-gradient oracle model of … Read more

    Long-Step Path-Following Algorithm for Solving Symmetric Programming Problems with Nonlinear Objective Functions

  • Leonid Faybusovich
  • Cunlu Zhou
    • Categories Convex Optimization, Linear, Cone and Semidefinite Programming Tags , ,

    We describe a long-step path-following algorithm for a class of symmetric programming problems with nonlinear convex objective functions. The complexity estimates similar to the case of a linear-quadratic objective function are established. The results of numerical experiments for the class of optimization problems involving quantum entropy are presented. CitationPreprint, University of Notre Dame, December 2017ArticleDownload … Read more

    Convergence Rates for Deterministic and Stochastic Subgradient Methods Without Lipschitz Continuity

  • Benjamin Grimmer
    • Categories Convex Optimization Tags , ,

    We generalize the classic convergence rate theory for subgradient methods to apply to non-Lipschitz functions via a new measure of steepness. For the deterministic projected subgradient method, we derive a global $O(1/\sqrt{T})$ convergence rate for any function with at most exponential growth. Our approach implies generalizations of the standard convergence rates for gradient descent on … Read more