Convex and Nonsmooth Optimization

A Modified Proximal Symmetric ADMM for Multi-Block Separable Convex Optimization with Linear Constraints

  • Yuan Shen
  • Xiayang Zhang
  • Yannian Zuo
    • Categories Convex Optimization Tags , , ,

    We consider the linearly constrained separable convex optimization problem whose objective function is separable w.r.t. $m$ blocks of variables. A bunch of methods have been proposed and well studied. Specifically, a modified strictly contractive Peaceman-Rachford splitting method (SC-PRCM) has been well studied in the literature for the special case of $m=3$. Based on the modified … Read more

    Optimization for Supervised Machine Learning: Randomized Algorithms for Data and Parameters

  • Filip Hanzely
    • Categories Convex Optimization, Stochastic Programming Tags , , , , ,

    Many key problems in machine learning and data science are routinely modeled as optimization problems and solved via optimization algorithms. With the increase of the volume of data and the size and complexity of the statistical models used to formulate these often ill-conditioned optimization tasks, there is a need for new efficient algorithms able to … Read more

    A geodesic interior-point method for linear optimization over symmetric cones

  • Frank Permenter
    • Categories Complementarity and Variational Inequalities, Convex Optimization, Linear, Cone and Semidefinite Programming Tags , ,

    We develop a new interior-point method (IPM) for symmetric-cone optimization, a common generalization of linear, second-order-cone, and semidefinite programming. In contrast to classical IPMs, we update iterates with a geodesic of the cone instead of the kernel of the linear constraints. This approach yields a primal-dual-symmetric, scale-invariant, and line-search-free algorithm that uses just half the … Read more

    A FISTA-type first order algorithm on composite optimization problems that is adaptable to the convex situation

  • Chee-Khian Sim
    • Categories Nonsmooth Optimization

    In this note, we propose a FISTA-type first order algorithm, VAR-FISTA, to solve a composite optimization problem. A distinctive feature of VAR-FISTA is its ability to exploit the convexity of the function in the problem, resulting in an improved iteration complexity when the function is convex compared to when it is nonconvex. The iteration complexity … Read more

    Split Bregman iteration for multi-period mean variance portfolio optimization

  • Stefania Corsaro
  • Valentina De Simone
  • Zelda Marino
    • Categories Applications - Science and Engineering, Convex and Nonsmooth Optimization Tags , , ,

    This paper investigates the problem of defining an optimal long-term investment strategy, where the investor can exit the investment before maturity without severe loss. Our setting is a multi-period one, where the aim is tomake a plan for allocating all of wealth among the n assets within a time horizon of m periods. In addition, … Read more

    Decentralized Learning with Lazy and Approximate Dual Gradients

  • Yanli Liu
  • Wotao Yin
  • Yuejiao Sun
    • Categories Convex Optimization

    This paper develops algorithms for decentralized machine learning over a network, where data are distributed, computation is localized, and communication is restricted between neighbors. A line of recent research in this area focuses on improving both computation and communication complexities. The methods SSDA and MSDA \cite{scaman2017optimal} have optimal communication complexity when the objective is smooth … Read more

    Iteration-complexity of an inner accelerated inexact proximal augmented Lagrangian method based on the classical Lagrangian function and a full Lagrange multiplier update

  • Jefferson Gonçalves Melo
  • Renato D.C. Monteiro
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    This paper establishes the iteration-complexity of an inner accelerated inexact proximal augmented Lagrangian (IAPIAL) method for solving linearly constrained smooth nonconvex composite optimization problems which is based on the classical Lagrangian function and, most importantly, performs a full Lagrangian multiplier update, i.e., no shrinking factor is incorporated on it. More specifically, each IAPIAL iteration consists … Read more

    Spatially Adaptive Regularization in Image Segmentation

  • Laura Antonelli
  • Valentina De Simone
  • Daniela di Serafino
    • Categories Applications - Science and Engineering, Convex and Nonsmooth Optimization

    We modify the total-variation-regularized image segmentation model proposed by Chan, Esedoglu and Nikolova [SIAM Journal on Applied Mathematics 66, 2006] by introducing local regularization that takes into account spatial image information. We propose some techniques for defining local regularization parameters, based on the cartoon-texture decomposition of the given image, on the mean and median filters, … Read more

    A Subspace Acceleration Method for Minimization Involving a Group Sparsity-Inducing Regularizer

  • Frank E. Curtis
  • Daniel P. Robinson
  • Yutong Dai
    • Categories Nonsmooth Optimization, Unconstrained Optimization Tags , , , , , , ,

    We consider the problem of minimizing an objective function that is the sum of a convex function and a group sparsity-inducing regularizer. Problems that integrate such regularizers arise in modern machine learning applications, often for the purpose of obtaining models that are easier to interpret and that have higher predictive accuracy. We present a new … Read more

    Accelerated Dual-Averaging Primal-Dual Method for Composite Convex Minimization

  • Shiqian Ma
  • Yuqiu Qian
  • Conghui Tan
  • Tong Zhang
    • Categories Convex and Nonsmooth Optimization

    Dual averaging-type methods are widely used in industrial machine learning applications due to their ability to promoting solution structure (e.g., sparsity) efficiently. In this paper, we propose a novel accelerated dual-averaging primal-dual algorithm for minimizing a composite convex function. We also derive a stochastic version of the proposed method which solves empirical risk minimization, and … Read more