Nonlinear Systems and Least-Squares

Alternating direction methods for non convex optimization with applications to second-order least-squares and risk parity portfolio selection

  • Xi Bai
  • Katya Scheinberg
    • Categories Finance and Economics, Nonlinear Optimization, Nonlinear Systems and Least-Squares Tags , , , , ,

    In this paper we mainly focus on optimization of sums of squares of quadratic functions, which we refer to as second-order least-squares problems, subject to convex constraints. Our motivation arises from applications in risk parity portfolio selection. We generalize the setting further by considering a class of nonlinear, non convex functions which admit a (non … Read more

    A Preconditioner for a Primal-Dual Newton Conjugate Gradients Method for Compressed Sensing Problems

  • Ioannis Dassios
  • Kimon Fountoulakis
  • Jacek Gondzio
    • Categories Nonlinear Systems and Least-Squares, Unconstrained Optimization Tags , , , ,

    In this paper we are concerned with the solution of Compressed Sensing (CS) problems where the signals to be recovered are sparse in coherent and redundant dictionaries. We extend a primal-dual Newton Conjugate Gradients (pdNCG) method for CS problems. We provide an inexpensive and provably effective preconditioning technique for linear systems using pdNCG. Numerical results … Read more

    Levenberg-Marquardt methods based on probabilistic gradient models and inexact subproblem solution, with application to data assimilation

  • Elhoucine Bergou
  • Serge Gratton
  • Luis Nunes Vicente
    • Categories Applications - Science and Engineering, Nonlinear Systems and Least-Squares, Stochastic Programming Tags , , , , , , ,

    The Levenberg-Marquardt algorithm is one of the most popular algorithms for the solution of nonlinear least squares problems. Motivated by the problem structure in data assimilation, we consider in this paper the extension of the classical Levenberg-Marquardt algorithm to the scenarios where the linearized least squares subproblems are solved inexactly and/or the gradient model is … Read more

    An Efficient Gauss-Newton Algorithm for Symmetric Low-Rank Product Matrix Approximations

  • Xin Liu
  • Zaiwen Wen
  • Yin Zhang
    • Categories Constrained Nonlinear Optimization, Nonlinear Systems and Least-Squares Tags , , ,

    We derive and study a Gauss-Newton method for computing a symmetric low-rank product that is the closest to a given symmetric matrix in Frobenius norm. Our Gauss-Newton method, which has a particularly simple form, shares the same order of iteration-complexity as a gradient method when the size of desired eigenspace is small, but can be … Read more

    A Fast Active Set Block Coordinate Descent Algorithm for l1-regularized least squares

  • Marianna De Santis
  • Stefano Lucidi
  • Francesco Rinaldi
    • Categories Convex and Nonsmooth Optimization, Nonlinear Systems and Least-Squares Tags , ,

    The problem of finding sparse solutions to underdetermined systems of linear equations arises in several real-world problems (e.g. signal and image processing, compressive sensing, statistical inference). A standard tool for dealing with sparse recovery is the l1-regularized least-squares approach that has been recently attracting the attention of many researchers. In this paper, we describe an … Read more

    Quasi-Newton updates with weighted secant equations

  • Serge Gratton
  • Vincent Malmedy
  • Philippe L. Toint
    • Categories Nonlinear Systems and Least-Squares, Unconstrained Optimization Tags , ,

    We provide a formula for variational quasi-Newton updates with multiple weighted secant equations. The derivation of the formula leads to a Sylvester equation in the correction matrix. Examples are given. CitationReport naXys-09-2013, Namur Centre for Complex Systems, Unibersity of Namur, Namur (Belgium)ArticleDownload View PDF

    On the Incomplete Oblique Projections Method for Solving Box Constrained Least Squares Problems

  • N. Echebest
  • MT Guardarucci
  • HD Scolnik
    • Categories Bound-constrained Optimization, Nonlinear Systems and Least-Squares Tags , ,

    The aim of this paper is to extend the applicability of the incomplete oblique projections method (IOP) previously introduced by the authors for solving inconsistent linear systems to the box constrained case. The new algorithm employs incomplete projections onto the set of solutions of the augmented system Ax-r= b, together with the box constraints, based … Read more

    Inexact Coordinate Descent: Complexity and Preconditioning

  • Jacek Gondzio
  • Peter Richtarik
  • Rachael Tappenden
    • Categories Convex Optimization, Nonlinear Systems and Least-Squares, Nonsmooth Optimization Tags , , , , ,

    In this paper we consider the problem of minimizing a convex function using a randomized block coordinate descent method. One of the key steps at each iteration of the algorithm is determining the update to a block of variables. Existing algorithms assume that in order to compute the update, a particular subproblem is solved exactly. … Read more

    Family Constraining of Iterative Algorithms

  • Yair Censor
  • Ioana Pantelimon
  • Constantin Popa
    • Categories Applications - Science and Engineering, Nonlinear Systems and Least-Squares Tags , ,

    In constraining iterative processes, the algorithmic operator of the iterative process is pre-multiplied by a constraining operator at each iterative step. This enables the constrained algorithm, besides solving the original problem, also to find a solution that incorporates some prior knowledge about the solution. This approach has been useful in image restoration and other image … Read more

    Adaptive Observations And Multilevel Optimization In Data Assimilation

  • Serge Gratton
  • Monserrat Rincon-Camacho
  • Philippe L. Toint
    • Categories Nonlinear Systems and Least-Squares, Optimization of Systems modeled by PDEs Tags , ,

    We propose to use a decomposition of large-scale incremental four dimensional (4D-Var) data assimilation problems in order to make their numerical solution more efficient. This decomposition is based on exploiting an adaptive hierarchy of the observations. Starting with a low-cardinality set and the solution of its corresponding optimization problem, observations are adaptively added based on … Read more