trust-region methods

Preconditioning and Globalizing Conjugate Gradients in Dual Space for Quadratically Penalized Nonlinear-Least Squares Problems

  • Serge Gratton
  • Philippe L. Toint
  • Selime Gürol
    • Categories Nonlinear Systems and Least-Squares, Optimization of Simulated Systems, Optimization of Systems modeled by PDEs Tags , , , , ,

    When solving nonlinear least-squares problems, it is often useful to regularize the problem using a quadratic term, a practice which is especially common in applications arising in inverse calculations. A solution method derived from a trust-region Gauss-Newton algorithm is analyzed for such applications, where, contrary to the standard algorithm, the least-squares subproblem solved at each … Read more

    Solving structured nonlinear least-squares and nonlinear feasibility problems with expensive functions

  • Markus Kaiser
  • Philippe L. Toint
  • Kathrin Klamroth
  • Alexander Thekale
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Nonlinear Systems and Least-Squares Tags , , , , , , ,

    We present an algorithm for nonlinear least-squares and nonlinear feasibility problems, i.e. for systems of nonlinear equations and nonlinear inequalities, which depend on the outcome of expensive functions for which derivatives are assumed to be unavailable. Our algorithm combines derivative-free techniques with filter trust-region methods to keep the number of expensive function evaluations low and … Read more

    Efficient Block-coordinate Descent Algorithms for the Group Lasso

  • Donald Goldfarb
  • Zhiwei (Tony) Qin
  • Katya Scheinberg
    • Categories Convex and Nonsmooth Optimization Tags , , , , ,

    We present two algorithms to solve the Group Lasso problem [Yuan & Lin]. First, we propose a general version of the Block Coordinate Descent (BCD) algorithm for the Group Lasso that employs an efficient approach for optimizing each subproblem. We show that it exhibits excellent performance when the groups are of moderate sizes. For large … Read more

    A Retrospective Filter Trust Region Algorithm For Unconstrained Optimization

  • Zhongwen Chen
  • Yue Lu
    • Categories Unconstrained Optimization Tags , , ,

    In this paper, we propose a retrospective filter trust region algorithm for unconstrained optimization, which is based on the framework of the retrospective trust region method and associated with the technique of the multi dimensional filter. The new algorithm gives a good estimation of trust region radius, relaxes the condition of accepting a trial step … Read more

    Bilevel Derivative-Free Optimization and its Application to Robust Optimization

  • Andrew R. Conn
  • Luis Nunes Vicente
    • Categories Nonlinear Optimization Tags , , , , ,

    We address bilevel programming problems when the derivatives of both the upper and the lower level objective functions are unavailable. The core algorithms used for both levels are trust-region interpolation-based methods, using minimum Frobenius norm quadratic models when the number of points is smaller than the number of basis components. We take advantage of the … Read more

    On affine scaling inexact dogleg methods for bound-constrained nonlinear systems

  • Stefania Bellavia
  • Sandra Pieraccini
  • Maria Macconi
    • Categories Nonlinear Systems and Least-Squares Tags , , , , ,

    A class of trust-region methods for large scale bound-constrained systems of nonlinear equations is presented. The methods in this class follow the so called affine-scaling approach and can efficiently handle large scale problems. At each iteration, a suitably scaled region around the current approximate solution is defined and, within such a region, the norm of … Read more

    Constrained Dogleg Methods for nonlinear systems with simple bounds

  • Stefania Bellavia
  • Sandra Pieraccini
  • Maria Macconi
    • Categories Nonlinear Systems and Least-Squares Tags , , , ,

    We focus on the numerical solution of medium scale bound-constrained systems of nonlinear equations. In this context, we consider an affine-scaling trust region approach that allows a great flexibility in choosing the scaling matrix used to handle the bounds. The method is based on a dogleg procedure tailored for constrained problems and so, it is … Read more

    A globally convergent modified conjugate-gradient line-search algorithm with inertia controlling

  • Ioannis G. Akrotirianakis
  • Joshua D. Griffin
  • Wenwen Zhou
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    In this paper we have addressed the problem of unboundedness in the search direction when the Hessian is indefinite or near singular. A new algorithm has been proposed which naturally handles singular Hessian matrices, and is theoretically equivalent to the trust-region approach. This is accomplished by performing explicit matrix modifications adaptively that mimic the implicit … Read more

    A Derivative-Free Algorithm for the Least-square minimization

  • Andrew R. Conn
  • Katya Scheinberg
  • Hongchao Zhang
    • Categories Nonlinear Optimization, Nonlinear Systems and Least-Squares Tags , , , , , ,

    We develop a framework for a class of derivative-free algorithms for the least-squares minimization problem. These algorithms are based on polynomial interpolation models and are designed to take advantages of the problem structure. Under suitable conditions, we establish the global convergence and local quadratic convergence properties of these algorithms. Promising numerical results indicate the algorithm … Read more

    A practical method for solving large-scale TRS

  • Marianna S. Apostolopoulou
  • C.A. Botsaris
  • Panagiotis Pintelas
  • Dimitris G. Sotiropoulos
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Quadratic Programming Tags , , , , ,

    We present a nearly-exact method for the large scale trust region subproblem (TRS) based on the properties of the minimal-memory BFGS method. Our study in concentrated in the case where the initial BFGS matrix can be any scaled identity matrix. The proposed method is a variant of the Mor\'{e}-Sorensen method that exploits the eigenstructure of … Read more