Convex and Nonsmooth Optimization

Optimality conditions for the nonlinear programming problems on Riemannian manifolds

  • Zhang Lei-Hong
  • Weihong Yang
  • Song Ruyi
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    In recent years, many traditional optimization methods have been successfully generalized to minimize objective functions on manifolds. In this paper, we first extend the general traditional constrained optimization problem to a nonlinear programming problem built upon a general Riemannian manifold $\mathcal{M}$, and discuss the first-order and the second-order optimality conditions. By exploiting the differential geometry … Read more

    A variable smoothing algorithm for solving convex optimization problems

  • Radu Ioan Bot
  • Christopher Hendrich
    • Categories Applications - Science and Engineering, Convex Optimization, Data-Mining Tags , , ,

    In this article we propose a method for solving unconstrained optimization problems with convex and Lipschitz continuous objective functions. By making use of the Moreau envelopes of the functions occurring in the objective, we smooth the latter to a convex and differentiable function with Lipschitz continuous gradient by using both variable and constant smoothing parameters. … Read more

    Aubin Property and Uniqueness of Solutions in Cone Constrained Optimization

  • Diethard Klatte
  • Bernd Kummer
    • Categories Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , ,

    We discuss conditions for the Aubin property of solutions to perturbed cone constrained programs, by using and refining results given in \cite{KlaKum02}. In particular, we show that constraint nondegeneracy and hence uniqueness of the multiplier is necessary for the Aubin property of the critical point map. Moreover, we give conditions under which the critical point … Read more

    A primal-dual splitting algorithm for finding zeros of sums of maximally monotone operators

  • Radu Ioan Bot
  • Ernö Robert Csetnek
  • Andre Heinrich
    • Categories Convex Optimization Tags , , , , ,

    We consider the primal problem of finding the zeros of the sum of a maximally monotone operator with the composition of another maximally monotone operator with a linear continuous operator and a corresponding dual problem formulated by means of the inverse operators. A primal-dual splitting algorithm which simultaneously solves the two problems in finite-dimensional spaces … Read more

    A Family of Second-Order Methods for Convex L1-Regularized Optimization

  • Richard H. Byrd
  • Gillian M Chin
  • Jorge Nocedal
  • Figen Oztoprak
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , ,

    This paper is concerned with the minimization of an objective that is the sum of a convex function $f$ and an $\ell_1$ regularization term. Our interest is in methods that incorporate second-order information about the function $f$ to accelerate convergence. We describe a semi-smooth Newton framework that can be used to generate a variety of … Read more

    A quasi-Newton proximal splitting method

  • Stephen Becker
  • Jalal Fadili
    • Categories Constrained Nonlinear Optimization, Convex Optimization, Nonsmooth Optimization

    A new result in convex analysis on the calculation of proximity operators in certain scaled norms is derived. We describe efficient implementations of the proximity calculation for a useful class of functions; the implementations exploit the piece-wise linear nature of the dual problem. The second part of the paper applies the previous result to acceleration … Read more

    Optimal Stochastic Approximation Algorithms for Strongly Convex Stochastic Composite Optimization, II: Shrinking Procedures and Optimal Algorithms

  • Saeed Ghadimi
  • Guanghui Lan
    • Categories Convex Optimization, Nonlinear Optimization, Stochastic Programming Tags , , , , ,

    In this paper we study new stochastic approximation (SA) type algorithms, namely, the accelerated SA (AC-SA), for solving strongly convex stochastic composite optimization (SCO) problems. Specifically, by introducing a domain shrinking procedure, we significantly improve the large-deviation results associated with the convergence rate of a nearly optimal AC-SA algorithm presented by the authors. Moreover, we … Read more

    Newton-Like Methods for Sparse Inverse Covariance Estimation

  • Jorge Nocedal
  • Peder Olsen
  • Figen Oztoprak
  • Stephen Rennie
    • Categories Convex Optimization Tags , ,

    We propose two classes of second-order optimization methods for solving the sparse inverse covariance estimation problem. The first approach, which we call the Newton-LASSO method, minimizes a piecewise quadratic model of the objective function at every iteration to generate a step. We employ the fast iterative shrinkage thresholding method (FISTA) to solve this subproblem. The … Read more

    Stochastic First- and Zeroth-order Methods for Nonconvex Stochastic Programming

  • Saeed Ghadimi
  • Guanghui Lan
    • Categories Convex and Nonsmooth Optimization, Stochastic Programming, Unconstrained Optimization Tags , , , ,

    In this paper, we introduce a new stochastic approximation (SA) type algorithm, namely the randomized stochastic gradient (RSG) method, for solving an important class of nonlinear (possibly nonconvex) stochastic programming (SP) problems. We establish the complexity of this method for computing an approximate stationary point of a nonlinear programming problem. We also show that this … Read more

    Greedy approximation in convex optimization

  • Vladimir Temlyakov
    • Categories Convex Optimization Tags , , ,

    We study sparse approximate solutions to convex optimization problems. It is known that in many engineering applications researchers are interested in an approximate solution of an optimization problem as a linear combination of elements from a given system of elements. There is an increasing interest in building such sparse approximate solutions using different greedy-type algorithms. … Read more