Nonlinear Optimization

An adaptive superfast inexact proximal augmented Lagrangian method for smooth nonconvex composite optimization problems

  • Arnesh Sujanani
  • Renato D.C. Monteiro
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , , , , ,

    This work presents an adaptive superfast proximal augmented Lagrangian (AS-PAL) method for solving linearly-constrained smooth nonconvex composite optimization problems. Each iteration of AS-PAL inexactly solves a possibly nonconvex proximal augmented Lagrangian (AL) subproblem obtained by an aggressive/adaptive choice of prox stepsize with the aim of substantially improving its computational performance followed by a full Lagrangian … Read more

    Asymptotic Consistency for Nonconvex Risk-Averse Stochastic Optimization with Infinite Dimensional Decision Spaces

  • Johannes Milz
  • Thomas Surowiec
    • Categories Optimization of Systems modeled by PDEs, Stochastic Programming, Systems governed by Differential Equations Optimization Tags , , , , , , ,

    Optimal values and solutions of empirical approximations of stochastic optimization problems can be viewed as statistical estimators of their true values. From this perspective, it is important to understand the asymptotic behavior of these estimators as the sample size goes to infinity, which is both of theoretical as well as practical interest. This area of … Read more

    Efficient composite heuristics for integer bound constrained noisy optimization

  • Morteza Kimiaei
  • Arnold Neumaier
    • Categories Bound-constrained Optimization, Integer Programming, Optimization Software Benchmark Tags , , , ,

    This paper discusses a composite algorithm for bound constrained noisy derivative-free optimization problems with integer variables. This algorithm is an integer variant of the matrix adaptation evolution strategy. An integer derivative-free line search strategy along affine scaling matrix directions is used to generate candidate points. Each affine scaling matrix direction is a product of the … Read more

    Worst-case evaluation complexity of a derivative-free quadratic regularization method

  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization Tags , , ,

    This short paper presents a derivative-free quadratic regularization method for unconstrained minimization of a smooth function with Lipschitz continuous gradient. At each iteration, trial points are computed by minimizing a quadratic regularization of a local model of the objective function. The models are based on forward finite-difference gradient approximations. By using a suitable acceptance condition … Read more

    Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution

  • Jianhao Ma
  • Salar Fattahi
    • Categories Nonlinear Optimization, Statistics Tags , ,

    This work characterizes the effect of depth on the optimization landscape of linear regression, showing that, despite their nonconvexity, deeper models have more desirable optimization landscape. We consider a robust and over-parameterized setting, where a subset of measurements are grossly corrupted with noise and the true linear model is captured via an $N$-layer linear neural … Read more

    The superiorization method with restarted perturbations for split minimization problems with an application to radiotherapy treatment planning

  • Francisco J. Aragón Artacho
  • Yair Censor
  • Aviv Gibali
  • David Torregrosa-Belén
    • Categories Biomedical Applications, Constrained Nonlinear Optimization, Convex Optimization Tags , , , , , ,

    In this paper we study the split minimization problem that consists of two constrained minimization problems in two separate spaces that are connected via a linear operator that maps one space into the other. To handle the data of such a problem we develop a superiorization approach that can reach a feasible point with reduced … Read more

    A Newton-CG based barrier method for finding a second-order stationary point of nonconvex conic optimization with complexity guarantees

  • Chuan He
  • Zhaosong Lu
    • Categories Linear, Cone and Semidefinite Programming, Nonlinear Optimization Tags , , , , ,

    In this paper we consider finding an approximate second-order stationary point (SOSP) of nonconvex conic optimization that minimizes a twice differentiable function over the intersection of an affine subspace and a convex cone. In particular, we propose a Newton-conjugate gradient (Newton-CG) based barrier method for finding an $(\epsilon,\sqrt{\epsilon})$-SOSP of this problem. Our method is not … Read more

    Accelerated gradient methods on the Grassmann and Stiefel manifolds

  • Xiaojing Zhu
    • Categories Constrained Nonlinear Optimization Tags , , , ,

    In this paper we extend a nonconvex Nesterov-type accelerated gradient (AG) method to optimization over the Grassmann and Stiefel manifolds. We propose an exponential-based AG algorithm for the Grassmann manifold and a retraction-based AG algorithm that exploits the Cayley transform for both of the Grassmann and Stiefel manifolds. Under some mild assumptions, we obtain the … Read more

    On the fulfillment of the complementary approximate Karush-Kuhn-Tucker conditions and algorithmic applications

  • Renan W. Prado
  • Sandra A. Santos
  • Lucas E. A. Simões
    • Categories Constrained Nonlinear Optimization Tags , ,

    Focusing on smooth constrained optimization problems, and inspired by the complementary approximate Karush-Kuhn-Tucker (CAKKT) conditions, this work introduces the weighted complementary Approximate Karush-Kuhn-Tucker (WCAKKT) conditions. They are shown to be verified not only by safeguarded augmented Lagrangian methods, but also by inexact restoration methods, inverse and logarithmic barrier methods, and a penalized algorithm for constrained … Read more

    A Quadratically Convergent Sequential Programming Method for Second-Order Cone Programs Capable of Warm Starts

  • Xinyi Luo
  • Andreas Wächter
    • Categories Constrained Nonlinear Optimization, Second-Order Cone Programming Tags , ,

    We propose a new method for linear second-order cone programs. It is based on the sequential quadratic programming framework for nonlinear programming. In contrast to interior point methods, it can capitalize on the warm-start capabilities of active-set quadratic programming subproblem solvers and achieve a local quadratic rate of convergence. In order to overcome the non-differentiability … Read more