Nonlinear Optimization

Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution

  • Jianhao Ma
  • Salar Fattahi
    • Categories Nonlinear Optimization, Statistics Tags , ,

    This work characterizes the effect of depth on the optimization landscape of linear regression, showing that, despite their nonconvexity, deeper models have more desirable optimization landscape. We consider a robust and over-parameterized setting, where a subset of measurements are grossly corrupted with noise and the true linear model is captured via an $N$-layer linear neural … Read more

    The superiorization method with restarted perturbations for split minimization problems with an application to radiotherapy treatment planning

  • Francisco J. Aragón Artacho
  • Yair Censor
  • Aviv Gibali
  • David Torregrosa-Belén
    • Categories Biomedical Applications, Constrained Nonlinear Optimization, Convex Optimization Tags , , , , , ,

    In this paper we study the split minimization problem that consists of two constrained minimization problems in two separate spaces that are connected via a linear operator that maps one space into the other. To handle the data of such a problem we develop a superiorization approach that can reach a feasible point with reduced … Read more

    A Newton-CG based barrier method for finding a second-order stationary point of nonconvex conic optimization with complexity guarantees

  • Chuan He
  • Zhaosong Lu
    • Categories Linear, Cone and Semidefinite Programming, Nonlinear Optimization Tags , , , , ,

    In this paper we consider finding an approximate second-order stationary point (SOSP) of nonconvex conic optimization that minimizes a twice differentiable function over the intersection of an affine subspace and a convex cone. In particular, we propose a Newton-conjugate gradient (Newton-CG) based barrier method for finding an $(\epsilon,\sqrt{\epsilon})$-SOSP of this problem. Our method is not … Read more

    Accelerated gradient methods on the Grassmann and Stiefel manifolds

  • Xiaojing Zhu
    • Categories Constrained Nonlinear Optimization Tags , , , ,

    In this paper we extend a nonconvex Nesterov-type accelerated gradient (AG) method to optimization over the Grassmann and Stiefel manifolds. We propose an exponential-based AG algorithm for the Grassmann manifold and a retraction-based AG algorithm that exploits the Cayley transform for both of the Grassmann and Stiefel manifolds. Under some mild assumptions, we obtain the … Read more

    On the fulfillment of the complementary approximate Karush-Kuhn-Tucker conditions and algorithmic applications

  • Renan W. Prado
  • Sandra A. Santos
  • Lucas E. A. Simões
    • Categories Constrained Nonlinear Optimization Tags , ,

    Focusing on smooth constrained optimization problems, and inspired by the complementary approximate Karush-Kuhn-Tucker (CAKKT) conditions, this work introduces the weighted complementary Approximate Karush-Kuhn-Tucker (WCAKKT) conditions. They are shown to be verified not only by safeguarded augmented Lagrangian methods, but also by inexact restoration methods, inverse and logarithmic barrier methods, and a penalized algorithm for constrained … Read more

    A Quadratically Convergent Sequential Programming Method for Second-Order Cone Programs Capable of Warm Starts

  • Xinyi Luo
  • Andreas Wächter
    • Categories Constrained Nonlinear Optimization, Second-Order Cone Programming Tags , ,

    We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence. In order to overcome the non-differentiability … Read more

    Hidden convexity in a class of optimization problems with bilinear terms

  • Bram L. Gorissen
  • Meike Reusken
  • Dick den Hertog
    • Categories Nonlinear Optimization, Robust Optimization Tags , , ,

    In this paper we identify a new class of nonconvex optimization problems that can be equivalently reformulated to convex ones. These nonconvex problems can be characterized by convex functions with bilinear arguments. We describe several examples of important applications that have this structure. These include the problems with variable coefficients, the dual of robust nonlinear … Read more

    An Adaptive Sampling Sequential Quadratic Programming Method for Equality Constrained Stochastic Optimization

  • Albert S. Berahas
  • Raghu Bollapragada
  • Baoyu Zhou
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , ,

    This paper presents a methodology for using varying sample sizes in sequential quadratic programming (SQP) methods for solving equality constrained stochastic optimization problems. The first part of the paper deals with the delicate issue of dynamic sample selection in the evaluation of the gradient in conjunction with inexact solutions to the SQP subproblems. Under reasonable … Read more

    On the weakest constraint qualification for sharp local minimizers

  • Oliver Stein
  • Maximilian Volk
    • Categories Constrained Nonlinear Optimization Tags , , , ,

    The sharp local minimality of feasible points of nonlinear optimization problems is known to possess a characterization by a strengthened version of the Karush-Kuhn-Tucker conditions, as long as the Mangasarian-Fromovitz constraint qualification holds. This strengthened condition is not easy to check algorithmically since it involves the topological interior of some set. In this paper we … Read more

    Convergence to a second-order critical point of composite nonsmooth problems by a trust region method

  • Hiroshi Yamashita
    • Categories Nonsmooth Optimization, Unconstrained Optimization Tags , , , ,

    An algorithm for finding a first-order and second-order critical point of composite nonsmooth problems is proposed in this paper. For smooth problems, algorithms for searching such a point usually utilize the so called negative-curvature directions. In this paper, the method recently proposed for nonlinear semidefinite problems by the current author is extended for solving general … Read more