Jennifer B. Erway

Shape-Changing Trust-Region Methods Using Multipoint Symmetric Secant Matrices

  • Johannes Brust
  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    In this work, we consider methods for large-scale and nonconvex unconstrained optimization. We propose a new trust-region method whose subproblem is defined using a so-called “shape-changing” norm together with densely-initialized multipoint symmetric secant (MSS) matrices to approximate the Hessian. Shape-changing norms and dense initializations have been successfully used in the context of traditional quasi Newton … Read more

    A New Multipoint Symmetric Secant Method with a Dense Initial Matrix

  • Jennifer B. Erway
  • Mostafa Rezapour
    • Categories Nonlinear Optimization

    In large-scale optimization, when either forming or storing Hessian matrices are prohibitively expensive, quasi-Newton methods are often used in lieu of Newton’s method because they only require first-order information to approximate the true Hessian.  Multipoint symmetric secant (MSS) methods can be thought of as generalizations of quasi-Newton methods in that they attempt to impose additional requirements on their approximation of the … Read more

    Trust-Region Algorithms for Training Responses: Machine Learning Methods Using Indefinite Hessian Approximations

  • Jennifer B. Erway
  • Joshua D. Griffin
  • Roummel F. Marcia
  • Riadh Omheni
    • Categories Applications - Science and Engineering, Unconstrained Optimization Tags , , ,

    Machine learning (ML) problems are often posed as highly nonlinear and nonconvex unconstrained optimization problems. Methods for solving ML problems based on stochastic gradient descent are easily scaled for very large problems but may involve fine-tuning many hyper-parameters. Quasi-Newton approaches based on the limited-memory Broyden-Fletcher-Goldfarb-Shanno (BFGS) update typically do not require manually tuning hyper-parameters but … Read more

    A Dense initialization for limited-memory quasi-Newton methods

  • Johannes Brust
  • Oleg Burdakov
  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    We consider a family of dense initializations for limited-memory quasi-Newton methods. The proposed initialization exploits an eigendecomposition-based separation of the full space into two complementary subspaces, assigning a different initialization parameter to each subspace. This family of dense initializations is proposed in the context of a limited-memory Broyden- Fletcher-Goldfarb-Shanno (L-BFGS) trust-region method that makes use … Read more

    ALGORITHM XXX: SC-SR1: MATLAB SOFTWARE FOR SOLVING SHAPE-CHANGING L-SR1 TRUST-REGION SUBPROBLEMS

  • Johannes Brust
  • Oleg Burdakov
  • Jennifer B. Erway
  • Roummel F. Marcia
  • Ya-xiang Yuan
    • Categories Unconstrained Optimization Tags , , , , ,

    We present a MATLAB implementation of the shape-changing sym- metric rank-one (SC-SR1) method that solves trust-region subproblems when a limited-memory symmetric rank-one (L-SR1) matrix is used in place of the true Hessian matrix. The method takes advantage of two shape-changing norms [4, 3] to decompose the trust-region subproblem into two separate problems. Using one of … Read more

    On solving large-scale limited-memory quasi-Newton equations

  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Nonlinear Optimization Tags , , , , , ,

    We consider the problem of solving linear systems of equations with limited- memory members of the restricted Broyden class and symmetric rank-one matrices. In this paper, we present various methods for solving these linear systems, and propose a new approach based on a practical implementation of the compact representation for the inverse of these limited-memory … Read more

    On efficiently computing the eigenvalues of limited-memory quasi-Newton matrices

  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Nonlinear Optimization Tags , , , ,

    In this paper, we consider the problem of efficiently computing the eigenvalues of limited-memory quasi-Newton matrices that exhibit a compact formulation. In addition, we produce a compact formula for quasi-Newton matrices generated by any member of the Broyden convex class of updates. Our proposed method makes use of efficient updates to the QR factorization that … Read more

    On Solving L-SR1 Trust-Region Subproblems

  • Johannes Brust
  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Optimization Software and Modeling Systems, Unconstrained Optimization Tags , , ,

    In this article, we consider solvers for large-scale trust-region subproblems when the quadratic model is defined by a limited-memory symmetric rank-one (L-SR1) quasi-Newton matrix. We propose a solver that exploits the compact representation of L-SR1 matrices. Our approach makes use of both an orthonormal basis for the eigenspace of the L-SR1 matrix and the Sherman- … Read more

    MSS: MATLAB software for L-BFGS trust-region subproblems for large-scale optimization

  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Nonlinear Optimization Tags , , ,

    A MATLAB implementation of the More’-Sorensen sequential (MSS) method is presented. The MSS method computes the minimizer of a quadratic function defined by a limited-memory BFGS matrix subject to a two-norm trust-region constraint. This solver is an adaptation of the More’-Sorensen direct method into an L-BFGS setting for large-scale optimization. The MSS method makes use … Read more

    A subspace minimization method for the trust-region step

  • Jennifer B. Erway
  • Philip E. Gill
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    We consider methods for large-scale unconstrained minimization based on finding an approximate minimizer of a quadratic function subject to a two-norm trust-region constraint. The Steihaug-Toint method uses the conjugate-gradient (CG) algorithm to minimize the quadratic over a sequence of expanding subspaces until the iterates either converge to an interior point or cross the constraint boundary. … Read more