nonconvex optimization

Sharp worst-case evaluation complexity bounds for arbitrary-order nonconvex optimization with inexpensive constraints

  • Coralia Cartis
  • Nicholas I. M. Gould
  • Philippe L. Toint
    • Categories Constrained Nonlinear Optimization, Data-Mining, Nonlinear Systems and Least-Squares Tags , , , , ,

    We provide sharp worst-case evaluation complexity bounds for nonconvex minimization problems with general inexpensive constraints, i.e.\ problems where the cost of evaluating/enforcing of the (possibly nonconvex or even disconnected) constraints, if any, is negligible compared to that of evaluating the objective function. These bounds unify, extend or improve all known upper and lower complexity bounds … Read more

    A Subsampling Line-Search Method with Second-Order Results

  • Elhoucine Bergou
  • Youssef Diouane
  • Vyacheslav Kungurtsev
  • Clément W. Royer
  • V. Kunc
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    In many contemporary optimization problems such as those arising in machine learning, it can be computationally challenging or even infeasible to evaluate an entire function or its derivatives. This motivates the use of stochastic algorithms that sample problem data, which can jeopardize the guarantees obtained through classical globalization techniques in optimization such as a trust … Read more

    Global Solutions of Nonconvex Standard Quadratic Programs via Mixed Integer Linear Programming Reformulations

  • Jacek Gondzio
  • E. Alper Yildirim
    • Categories (Mixed) Integer Linear Programming, Quadratic Programming Tags , , ,

    A standard quadratic program is an optimization problem that consists of minimizing a (nonconvex) quadratic form over the unit simplex. We focus on reformulating a standard quadratic program as a mixed integer linear programming problem. We propose two alternative mixed integer linear programming formulations. Our first formulation is based on casting a standard quadratic program … Read more

    Second-order Guarantees of Distributed gradient Algorithms

  • Amir Daneshmand
  • Vyacheslav Kungurtsev
  • Gesualdo Scutari
    • Categories Nonlinear Optimization Tags , ,

    We consider distributed smooth nonconvex unconstrained optimization over networks, modeled as a connected graph. We examine the behavior of distributed gradient-based algorithms near strict saddle points. Specifically, we establish that (i) the renowned Distributed Gradient Descent (DGD) algorithm likely converges to a neighborhood of a Second-order Stationary (SoS) solution; and (ii) the more recent class … Read more

    Convergence rates for an inertial algorithm of gradient type associated to a smooth nonconvex minimization

  • Laszlo Szilard Csaba
    • Categories Other Topics Tags , , ,

    We investigate an inertial algorithm of gradient type in connection with the minimization of a nonconvex differentiable function. The algorithm is formulated in the spirit of Nesterov’s accelerated convex gradient method. We show that the generated sequences converge to a critical point of the objective function, if a regularization of the objective function satis es the … Read more

    Efficient Optimization Algorithms for Robust Principal Component Analysis and Its Variants

  • Necdet Serhat Aybat
  • Shiqian Ma
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , , , ,

    Robust PCA has drawn significant attention in the last decade due to its success in numerous application domains, ranging from bio-informatics, statistics, and machine learning to image and video processing in computer vision. Robust PCA and its variants such as sparse PCA and stable PCA can be formulated as optimization problems with exploitable special structures. … Read more

    Parallel and Distributed Successive Convex Approximation Methods for Big-Data Optimization

  • Gesualdo Scutari
  • Ying Sun
    • Categories Applications - Science and Engineering, Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , ,

    Recent years have witnessed a surge of interest in parallel and distributed optimization methods for large-scale systems. In particular, nonconvex large-scale optimization problems have found a wide range of applications in several engineering fields. The design and the analysis of such complex, large-scale, systems pose several challenges and call for the development of new optimization … Read more

    A second order dynamical approach with variable damping to nonconvex smooth minimization

  • Radu Ioan Bot
  • Ernö Robert Csetnek
  • Szilard Csaba Laszlo
    • Categories Nonsmooth Optimization Tags , , ,

    We investigate a second order dynamical system with variable damping in connection with the minimization of a nonconvex differentiable function. The dynamical system is formulated in the spirit of the differential equation which models Nesterov’s accelerated convex gradient method. We show that the generated trajectory converges to a critical point, if a regularization of the … Read more

    A comparison of methods for traversing non-convex regions in optimization problems

  • Michael Bartholomew-Biggs
  • Salah Beddiaf
  • Bruce Christianson
    • Categories Unconstrained Optimization Tags , ,

    This paper considers again the well-known problem of dealing with non-convex regions during the minimization of a nonlinear function F(x) by Newton-like methods. The proposal made here involves a curvilinear search along an approximation to the continuous steepest descent path defined by the solution of the ODE dx/dt = -grad F(x). The algorithm we develop … Read more

    Strong Convex Nonlinear Relaxations of the Pooling Problem

  • Claudia D'Ambrosio
  • Jeff Linderoth
  • James R. Luedtke
  • Jonas Schweiger
    • Categories Chemical Engineering, Cutting Plane Approaches, Global Optimization Tags , ,

    We investigate new convex relaxations for the pooling problem, a classic nonconvex production planning problem in which input materials are mixed in intermediate pools, with the outputs of these pools further mixed to make output products meeting given attribute percentage requirements. Our relaxations are derived by considering a set which arises from the formulation by … Read more