trust-region methods

Improving the Flexibility and Robustness of Model-Based Derivative-Free Optimization Solvers

  • Coralia Cartis
  • Jan Fiala
  • Benjamin Marteau
  • Lindon Roberts
    • Categories Nonlinear Optimization Tags , , , , ,

    We present DFO-LS, a software package for derivative-free optimization (DFO) for nonlinear Least-Squares (LS) problems, with optional bound constraints. Inspired by the Gauss-Newton method, DFO-LS constructs simplified linear regression models for the residuals. DFO-LS allows flexible initialization for expensive problems, whereby it can begin making progress from as few as two objective evaluations. Numerical results … Read more

    On an Elliptical Trust-Region Procedure for Ill-Posed Nonlinear Least-Squares Problems

  • Stefania Bellavia
  • Elisa Riccietti
    • Categories Nonlinear Systems and Least-Squares Tags , , , ,

    In this paper we address the stable numerical solution of ill-posed nonlinear least-squares problems with small residual. We propose an elliptical trust-region reformulation of a Levenberg-Marquardt procedure. Thanks to an appropriate choice of the trust-region radius, the proposed procedure guarantees an automatic choice of the free regularization parameters that, together with a suitable stopping criterion, … Read more

    A Trust Region Algorithm for Heterogeneous Multiobjective Optimization

  • Gabriele Eichfelder
  • Jana Thomann
    • Categories Multi-Criteria Optimization Tags , , , ,

    This paper presents a new trust region method for multiobjective heterogeneous optimization problems. One of the objective functions is an expensive black-box function, for example given by a time-consuming simulation. For this function derivative information cannot be used and the computation of function values involves high computational effort. The other objective functions are given analytically … Read more

    Concise Complexity Analyses for Trust-Region Methods

  • Frank E. Curtis
  • Daniel P. Robinson
  • Zachary Lubberts
    • Categories Unconstrained Optimization Tags , , , , , ,

    Concise complexity analyses are presented for simple trust region algorithms for solving unconstrained optimization problems. In contrast to a traditional trust region algorithm, the algorithms considered in this paper require certain control over the choice of trust region radius after any successful iteration. The analyses highlight the essential algorithm components required to obtain certain complexity … Read more

    FINITE ELEMENT MODEL UPDATING FOR STRUCTURAL APPLICATIONS

  • Maria Girardi
  • Margherita Porcelli
  • Cristina Padovani
  • Daniele Pellegrini
  • Leonardo Robol
    • Categories Civil and Environmental Engineering, Nonlinear Optimization Tags , , , ,

    A novel method for performing model updating on finite element models is presented. The approach is particularly tailored to modal analyses of buildings, by which the lowest frequencies, obtained by using sensors and system identification approaches, need to be matched to the numerical ones predicted by the model. This is done by optimizing some unknown … Read more

    Regional Complexity Analysis of Algorithms for Nonconvex Smooth Optimization

  • Frank E. Curtis
  • Daniel P. Robinson
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , , ,

    A strategy is proposed for characterizing the worst-case performance of algorithms for solving nonconvex smooth optimization problems. Contemporary analyses characterize worst-case performance by providing, under certain assumptions on an objective function, an upper bound on the number of iterations (or function or derivative evaluations) required until a pth-order stationarity condition is approximately satisfied. This arguably … Read more

    A Stochastic Trust Region Algorithm Based on Careful Step Normalization

  • Frank E. Curtis
  • Katya Scheinberg
  • Rui Shi
    • Categories Nonlinear Optimization, Stochastic Programming, Unconstrained Optimization Tags , , , , , ,

    An algorithm is proposed for solving stochastic and finite sum minimization problems. Based on a trust region methodology, the algorithm employs normalized steps, at least as long as the norms of the stochastic gradient estimates are within a specified interval. The complete algorithm—which dynamically chooses whether or not to employ normalized steps—is proved to have … Read more

    A derivative-free Gauss-Newton method

  • Coralia Cartis
  • Lindon Roberts
    • Categories Nonlinear Optimization Tags , , , , ,

    We present DFO-GN, a derivative-free version of the Gauss-Newton method for solving nonlinear least-squares problems. As is common in derivative-free optimization, DFO-GN uses interpolation of function values to build a model of the objective, which is then used within a trust-region framework to give a globally-convergent algorithm requiring $O(\epsilon^{-2})$ iterations to reach approximate first-order criticality … Read more

    A Dense initialization for limited-memory quasi-Newton methods

  • Johannes Brust
  • Oleg Burdakov
  • Jennifer B. Erway
  • Roummel F. Marcia
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    We consider a family of dense initializations for limited-memory quasi-Newton methods. The proposed initialization exploits an eigendecomposition-based separation of the full space into two complementary subspaces, assigning a different initialization parameter to each subspace. This family of dense initializations is proposed in the context of a limited-memory Broyden- Fletcher-Goldfarb-Shanno (L-BFGS) trust-region method that makes use … Read more

    Globally Solving the Trust Region Subproblem Using Simple First-Order Methods

  • Amir Beck
  • Yakov Vaisbourd
    • Categories Constrained Nonlinear Optimization Tags , , , ,

    We consider the trust region subproblem which is given by a minimization of a quadratic, not necessarily convex, function over the Euclidean ball. Based on the well-known second-order necessary and sufficient optimality conditions for this problem, we present two sufficient optimality conditions defined solely in terms of the primal variables. Each of these conditions corresponds … Read more