composite optimization

An Inexact Spingarn’s Partial Inverse Method with Applications to Operator Splitting and Composite Optimization

  • M. Marques Alves
  • Samara Costa Lima
    • Categories Convex and Nonsmooth Optimization Tags , , , , , ,

    We propose and study the iteration-complexity of an inexact version of the Spingarn’s partial inverse method. Its complexity analysis is performed by viewing it in the framework of the hybrid proximal extragradient (HPE) method, for which pointwise and ergodic iteration-complexity has been established recently by Monteiro and Svaiter. As applications, we propose and analyze the … Read more

    A Multilevel Proximal Gradient Algorithm for a Class of Composite Optimization Problems

  • Panos Parpas
    • Categories Nonlinear Optimization Tags , ,

    Composite optimization models consist of the minimization of the sum of a smooth (not necessarily convex) function and a non-smooth convex function. Such models arise in many applications where, in addition to the composite nature of the objective function, a hierarchy of models is readily available. It is common to take advantage of this hierarchy … Read more

    An Asynchronous Mini-Batch Algorithm for Regularized Stochastic Optimization

  • Arda Aytekin
  • Hamid Reza Feyzmahdavian
  • Mikael Johansson
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , ,

    Mini-batch optimization has proven to be a powerful paradigm for large-scale learning. However, the state of the art parallel mini-batch algorithms assume synchronous operation or cyclic update orders. When worker nodes are heterogeneous (due to different computational capabilities or different communication delays), synchronous and cyclic operations are inefficient since they will leave workers idle waiting … Read more

    Composite Self-concordant Minimization

  • Volkan Cevher
  • Quoc Tran-Dinh
  • Anastasios Kyrillidis
    • Categories Convex and Nonsmooth Optimization, Convex Optimization, Statistics Tags , , , ,

    We propose a variable metric framework for minimizing the sum of a self-concordant function and a possibly non-smooth convex function endowed with a computable proximal operator. We theoretically establish the convergence of our framework without relying on the usual Lipschitz gradient assumption on the smooth part. An important highlight of our work is a new … Read more

    Randomized Block Coordinate Non-Monotone Gradient Method for a Class of Nonlinear Programming

  • Zhaosong Lu
  • Lin Xiao
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , ,

    In this paper we propose a randomized block coordinate non-monotone gradient (RBCNMG) method for minimizing the sum of a smooth (possibly nonconvex) function and a block-separable (possibly nonconvex nonsmooth) function. At each iteration, this method randomly picks a block according to any prescribed probability distribution and typically solves several associated proximal subproblems that usually have … Read more

    On the Complexity Analysis of Randomized Block-Coordinate Descent Methods

  • Zhaosong Lu
  • Lin Xiao
    • Categories Convex and Nonsmooth Optimization Tags , , , ,

    In this paper we analyze the randomized block-coordinate descent (RBCD) methods for minimizing the sum of a smooth convex function and a block-separable convex function. In particular, we extend Nesterov’s technique (SIOPT 2012) for analyzing the RBCD method for minimizing a smooth convex function over a block-separable closed convex set to the aforementioned more general … Read more

    Iteration Complexity of Randomized Block-Coordinate Descent Methods for Minimizing a Composite Function

  • Peter Richtarik
  • Martin Takac
    • Categories Convex Optimization Tags , , , , , , ,

    In this paper we develop a randomized block-coordinate descent method for minimizing the sum of a smooth and a simple nonsmooth block-separable convex function and prove that it obtains an $\epsilon$-accurate solution with probability at least $1-\rho$ in at most $O(\tfrac{n}{\epsilon} \log \tfrac{1}{\rho})$ iterations, where $n$ is the number of blocks. For strongly convex functions … Read more

    An Approximate Lagrange Multiplier Rule

  • Joydeep Dutta
  • Suvendu Pattanaik
  • Michel Thera
    • Categories Nonsmooth Optimization Tags , , , ,

    In this paper, we show that for a large class of optimization problems, the Lagrange multiplier rule can be derived from the so-called approximate multiplier rule. In establishing the link between the approximate and the exact multiplier rule we first derive an approximate multiplier rule for a very general class of optimization problems using the … Read more

    Identifying Activity

  • Adrian Lewis
  • Stephen Wright
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    Identification of active constraints in constrained optimization is of interest from both practical and theoretical viewpoints, as it holds the promise of reducing an inequality-constrained problem to an equality-constrained problem, in a neighborhood of a solution. We study this issue in the more general setting of composite nonsmooth minimization, in which the objective is a … Read more