Convex and Nonsmooth Optimization

Convergence rate bounds for a proximal ADMM with over-relaxation stepsize parameter for solving nonconvex linearly constrained problems

  • Max L. N. Gonçalves
  • Jefferson Gonçalves Melo
  • Renato D.C. Monteiro
    • Categories Convex and Nonsmooth Optimization, Nonsmooth Optimization Tags , , ,

    This paper establishes convergence rate bounds for a variant of the proximal alternating direction method of multipliers (ADMM) for solving nonconvex linearly constrained optimization problems. The variant of the proximal ADMM allows the inclusion of an over-relaxation stepsize parameter belonging to the interval (0,2). To the best of our knowledge, all related papers in the … Read more

    Convex Optimization with ALADIN

  • Moritz Diehl
  • Boris Houska
  • Yuning Jiang
  • Dimitris Kouzoupis
    • Categories Convex Optimization Tags ,

    This paper presents novel convergence results for the Augmented Lagrangian based Alternating Direction Inexact Newton method (ALADIN) in the context of distributed convex optimization. It is shown that ALADIN converges for a large class of convex optimization problems from any starting point to minimizers without needing line-search or other globalization routines. Under additional regularity assumptions, … Read more

    Asynchronous Parallel Algorithms for Nonconvex Big-Data Optimization. Part I: Model and Convergence

  • Loris Cannelli
  • Francisco Facchinei
  • Vyacheslav Kungurtsev
  • Gesualdo Scutari
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Parallel Algorithms Tags , , ,

    We propose a novel asynchronous parallel algorithmic framework for the minimization of the sum of a smooth nonconvex function and a convex nonsmooth regularizer, subject to both convex and nonconvex constraints. The proposed framework hinges on successive convex approximation techniques and a novel probabilistic model that captures key elements of modern computational architectures and asynchronous … Read more

    Asynchronous Parallel Algorithms for Nonconvex Big-Data Optimization. Part II: Complexity and Numerical Results

  • Loris Cannelli
  • Francisco Facchinei
  • Vyacheslav Kungurtsev
  • Gesualdo Scutari
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Parallel Algorithms Tags , , ,

    We present complexity and numerical results for a new asynchronous parallel algorithmic method for the minimization of the sum of a smooth nonconvex function and a convex nonsmooth regularizer, subject to both convex and nonconvex constraints. The proposed method hinges on successive convex approximation techniques and a novel probabilistic model that captures key elements of … Read more

    Communication-Efficient Algorithms for Decentralized and Stochastic Optimization

  • Guanghui Lan
  • Soomin Lee
  • Yi Zhou
    • Categories Convex and Nonsmooth Optimization, Stochastic Programming Tags , , , , , ,

    We present a new class of decentralized first-order methods for nonsmooth and stochastic optimization problems defined over multiagent networks. Considering that communication is a major bottleneck in decentralized optimization, our main goal in this paper is to develop algorithmic frameworks which can significantly reduce the number of inter-node communications. We first propose a decentralized primal-dual … Read more

    Quantitative Stability Analysis for Minimax Distributionally Robust RiskOptimization

  • Alois Pichler
  • Huifu Xu
    • Categories Convex and Nonsmooth Optimization, Stochastic Programming Tags , ,

    This paper considers distributionally robust formulations of a two stage stochastic programming problem with the objective of minimizing a distortion risk of the minimal cost incurred at the second stage. We carry out stability analysis by looking into variations of the ambiguity set under the Wasserstein metric, decision spaces at both stages and the support … Read more

    On level regularization with normal solutions in decomposition methods for multistage stochastic programming problems

  • Welington de Oliveira
  • Yongjia Song
  • Wim van Ackooij
    • Categories Nonsmooth Optimization, Stochastic Programming Tags , , ,

    We consider well-known decomposition techniques for multistage stochastic programming and a new scheme based on normal solutions for stabilizing iterates during the solution process. The given algorithms combine ideas from finite perturbation of convex programs and level bundle methods to regularize the so-called forward step of these decomposition methods. Numerical experiments on a hydrothermal scheduling … Read more

    Geometric descent method for convex composite minimization

  • Shixiang Chen
  • Wei Liu
  • Shiqian Ma
    • Categories Convex and Nonsmooth Optimization

    In this paper, we extend the geometric descent method recently proposed by Bubeck, Lee and Singh to tackle nonsmooth and strongly convex composite problems. We prove that our proposed algorithm, dubbed geometric proximal gradient method (GeoPG), converges with a linear rate $(1-1/\sqrt{\kappa})$ and thus achieves the optimal rate among first-order methods, where $\kappa$ is the … Read more

    An extension of Chubanov’s algorithm to symmetric cones

  • Tomonari Kitahara
  • Bruno F. Lourenço
  • Masakazu Muramatsu
  • Takashi Tsuchiya
    • Categories Convex and Nonsmooth Optimization, Linear, Cone and Semidefinite Programming Tags , ,

    In this work we present an extension of Chubanov’s algorithm to the case of homogeneous feasibility problems over a symmetric cone K. As in Chubanov’s method for linear feasibility problems, the algorithm consists of a basic procedure and a step where the solutions are confined to the intersection of a half-space and K. Following an … Read more