Nonlinear Optimization

Adaptive Augmented Lagrangian Methods: Algorithms and Practical Numerical Experience

  • Frank E. Curtis
  • Nicholas I. M. Gould
  • Daniel P. Robinson
  • Hao Jiang
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , , , ,

    In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis et al. [An adaptive augmented Lagrangian method for large-scale constrained optimization, Math. Program. 152 (2015), pp.201–245.]. … Read more

    Quadratic regularization projected alternating Barzilai–Borwein method for constrained optimization

  • Yakui Huang
  • Hongwei Liu
  • Sha Zhou
    • Categories Applications - Science and Engineering, Nonlinear Optimization Tags , , , ,

    In this paper, based on the regularization techniques and projected gradient strategies, we present a quadratic regularization projected alternating Barzilai–Borwein (QRPABB) method for minimizing differentiable functions on closed convex sets. We show the convergence of the QRPABB method to a constrained stationary point for a nonmonotone line search. When the objective function is convex, we … Read more

    Preconditioning of Active-Set Newton Methods for PDE-constrained Optimal Control Problems

  • Margherita Porcelli
  • Valeria Simoncini
  • Mattia Tani
    • Categories Control Applications, Nonlinear Optimization, Systems governed by Differential Equations Optimization Tags , ,

    We address the problem of preconditioning a sequence of saddle point linear systems arising in the solution of PDE-constrained optimal control problems via active-set Newton methods, with control and (regularized) state constraints. We present two new preconditioners based on a full block matrix factorization of the Schur complement of the Jacobian matrices, where the active-set … Read more

    On Second Order Optimality Conditions in Nonlinear Optimization

  • Roberto Andreani
  • Roger Behling
  • Gabriel Haeser
  • Paulo J. S. Silva
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    In this work we present new weak conditions that ensure the validity of necessary second order optimality conditions (SOC) for nonlinear optimization. We are able to prove that weak and strong SOCs hold for all Lagrange multipliers using Abadie-type assumptions. We also prove weak and strong SOCs for at least one Lagrange multiplier imposing the … Read more

    Block stochastic gradient iteration for convex and nonconvex optimization

  • Yangyang Xu
  • Wotao Yin
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Stochastic Approaches Tags , ,

    The stochastic gradient (SG) method can minimize an objective function composed of a large number of differentiable functions, or solve a stochastic optimization problem, to a moderate accuracy. The block coordinate descent/update (BCD) method, on the other hand, handles problems with multiple blocks of variables by updating them one at a time; when the blocks … Read more

    On efficiency of nonmonotone Armijo-type line searches

  • Masoud Ahookhosh
  • Susan Ghaderi
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , , ,

    Monotonicity and nonmonotonicity play a key role in studying the global convergence and the efficiency of iterative schemes employed in the field of nonlinear optimization, where globally convergent and computationally efficient schemes are explored. This paper addresses some features of descent schemes and the motivation behind nonmonotone strategies and investigates the efficiency of an Armijo-type … Read more

    A proximal point algorithm for DC functions on Hadamard manifolds

  • Paulo Roberto Oliveira
  • João Carlos Souza
    • Categories Other Topics, Unconstrained Optimization Tags

    An extension of a proximal point algorithm for difference of two convex functions is presented in the context of Riemannian manifolds of nonposite sectional curvature. If the sequence generated by our algorithm is bounded it is proved that every cluster point is a critical point of the function (not necessarily convex) under consideration, even if … Read more

    An improved algorithm for L2-Lp minimization problem

  • Dongdong Ge
  • He Rongchuan
  • He Simai
    • Categories Global Optimization Theory, Nonsmooth Optimization, Unconstrained Optimization

    In this paper we consider a class of non-Lipschitz and non-convex minimization problems which generalize the L2−Lp minimization problem. We propose an iterative algorithm that decides the next iteration based on the local convexity/concavity/sparsity of its current position. We show that our algorithm finds an epsilon-KKT point within O(log(1/epsilon)) iterations. The same result is also … Read more

    A regularized limited-memory BFGS method for unconstrained minimization problems

  • Shinji Sugimoto
  • Nobuo Yamashita
    • Categories Unconstrained Optimization Tags , ,

    The limited-memory BFGS (L-BFGS) algorithm is a popular method of solving large-scale unconstrained minimization problems. Since L-BFGS conducts a line search with the Wolfe condition, it may require many function evaluations for ill-posed problems. To overcome this difficulty, we propose a method that combines L-BFGS with the regularized Newton method. The computational cost for a … Read more

    A Branch-and-Bound Algorithm for Instrumental Variable Quantile Regression

  • Samuel Burer
  • Guanglin Xu
    • Categories Nonlinear Optimization, Quadratic Programming, Statistics

    This paper studies a statistical problem called instrumental variable quantile regres- sion (IVQR). We model IVQR as a convex quadratic program with complementarity constraints and—although this type of program is generally NP-hard—we develop a branch-and-bound algorithm to solve it globally. We also derive bounds on key vari- ables in the problem, which are valid asymptotically … Read more