augmented lagrangian method

Linearized Alternating Direction Method of Multipliers via Positive-Indefinite Proximal Regularization for Convex Programming

  • Bingsheng He
  • Feng Ma
  • Xiao-Ming Yuan
    • Categories Convex Optimization Tags , , , ,

    The alternating direction method of multipliers (ADMM) is being widely used for various convex minimization models with separable structures arising in a variety of areas. In the literature, the proximalversion of ADMM which allows ADMM’s subproblems to be proximally regularized has been well studied. Particularly the linearized version of ADMM can be yielded when the … Read more

    A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems

  • Xudong Li
  • Defeng Sun
  • Kim-Chuan Toh
    • Categories Nonsmooth Optimization Tags , , , ,

    We develop a fast and robust algorithm for solving large scale convex composite optimization models with an emphasis on the $\ell_1$-regularized least squares regression (Lasso) problems. Despite the fact that there exist a large number of solvers in the literature for the Lasso problems, we found that no solver can efficiently handle difficult large scale … Read more

    Global Convergence of ADMM in Nonconvex Nonsmooth Optimization

  • Yu Wang
  • Wotao Yin
  • Jinshan Zeng
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , , ,

    In this paper, we analyze the convergence of the alternating direction method of multipliers (ADMM) for minimizing a nonconvex and possibly nonsmooth objective function, $\phi(x_1,\ldots,x_p,y)$, subject to linear equality constraints that couple $x_1,\ldots,x_p,y$, where $p\ge 1$ is an integer. Our ADMM sequentially updates the primal variables in the order $x_1,\ldots,x_p,y$, followed by updating the dual … Read more

    Sparse Recovery via Partial Regularization: Models, Theory and Algorithms

  • Xiaorui Li
  • Zhaosong Lu
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Statistics Tags , , , , , , ,

    In the context of sparse recovery, it is known that most of existing regularizers such as $\ell_1$ suffer from some bias incurred by some leading entries (in magnitude) of the associated vector. To neutralize this bias, we propose a class of models with partial regularizers for recovering a sparse solution of a linear system. We … Read more

    Sequential equality-constrained optimization for nonlinear programming

  • Ernesto G. Birgin
  • Luís Felipe Bueno
  • José Mario Martínez
    • Categories Constrained Nonlinear Optimization Tags , , ,

    A new method is proposed for solving optimization problems with equality constraints and bounds on the variables. In the spirit of Sequential Quadratic Programming and Sequential Linearly-Constrained Programming, the new method approximately solves, at each iteration, an equality-constrained optimization problem. The bound constraints are handled in outer iterations by means of an Augmented Lagrangian scheme. … Read more

    Alternating direction methods for non convex optimization with applications to second-order least-squares and risk parity portfolio selection

  • Xi Bai
  • Katya Scheinberg
    • Categories Finance and Economics, Nonlinear Optimization, Nonlinear Systems and Least-Squares Tags , , , , ,

    In this paper we mainly focus on optimization of sums of squares of quadratic functions, which we refer to as second-order least-squares problems, subject to convex constraints. Our motivation arises from applications in risk parity portfolio selection. We generalize the setting further by considering a class of nonlinear, non convex functions which admit a (non … Read more

    How the augmented Lagrangian algorithm can deal with an infeasible convex quadratic optimization problem

  • Alice Chiche
  • Jean Charles Gilbert
    • Categories Convex and Nonsmooth Optimization, Quadratic Programming Tags , , , , , , , , ,

    This paper analyses the behavior of the augmented Lagrangian algorithm when it deals with an infeasible convex quadratic optimization problem. It is shown that the algorithm finds a point that, on the one hand, satisfies the constraints shifted by the smallest possible shift that makes them feasible and, on the other hand, minimizes the objective … Read more

    Adaptive Augmented Lagrangian Methods: Algorithms and Practical Numerical Experience

  • Frank E. Curtis
  • Nicholas I. M. Gould
  • Daniel P. Robinson
  • Hao Jiang
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , , , ,

    In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis et al. [An adaptive augmented Lagrangian method for large-scale constrained optimization, Math. Program. 152 (2015), pp.201–245.]. … Read more

    A Globally Convergent Stabilized SQP Method: Superlinear Convergence

  • Philip E. Gill
  • Vyacheslav Kungurtsev
  • Daniel P. Robinson
    • Categories Constrained Nonlinear Optimization Tags , , , , , ,

    Regularized and stabilized sequential quadratic programming (SQP) methods are two classes of methods designed to resolve the numerical and theoretical difficulties associated with ill-posed or degenerate nonlinear optimization problems. Recently, a regularized SQP method has been proposed that allows convergence to points satisfying certain second-order KKT conditions (SIAM J. Optim., 23(4):1983–2010, 2013). The method is … Read more

    A note on Fejér-monotone sequences in product spaces and its applications to the dual convergence of augmented Lagrangian methods

  • M. Marques Alves
  • Benar Fux Svaiter
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , ,

    In a recent Math. Program. paper, Eckstein and Silva proposed a new error criterion for the approximate solutions of augmented Lagrangian subproblems. Based on a saddle-point formulation of the primal and dual problems, they proved that dual sequences generated by augmented Lagrangians under this error criterion are bounded and that theirs limit points are dual … Read more