regularization

Cubic Regularization Method based on Mixed Factorizations for Unconstrained Minimization

  • Ernesto G. Birgin
  • José Mario Martínez
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    Newton’s method for unconstrained optimization, subject to proper regularization or special trust-region procedures, finds first-order stationary points with precision $\varepsilon$ employing, at most, $O(\varepsilon^{-3/2})$ functional and derivative evaluations. However, the computer work per iteration of the best-known implementations may need several factorizations per iteration or may use rather expensive matrix decompositions. In this paper, we … Read more

    Regional Complexity Analysis of Algorithms for Nonconvex Smooth Optimization

  • Frank E. Curtis
  • Daniel P. Robinson
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , , ,

    A strategy is proposed for characterizing the worst-case performance of algorithms for solving nonconvex smooth optimization problems. Contemporary analyses characterize worst-case performance by providing, under certain assumptions on an objective function, an upper bound on the number of iterations (or function or derivative evaluations) required until a pth-order stationarity condition is approximately satisfied. This arguably … Read more

    On the use of third-order models with fourth-order regularization for unconstrained optimization

  • Ernesto G. Birgin
  • José Mario Martínez
  • Sandra Augusta Santos
  • John L. Gardenghi
    • Categories Unconstrained Optimization Tags , , ,

    In a recent paper, it was shown that, for the smooth unconstrained optimization problem, worst-case evaluation complexity $O(\epsilon^{-(p+1)/p})$ may be obtained by means of algorithms that employ sequential approximate minimizations of p-th order Taylor models plus (p + 1)-th order regularization terms. The aforementioned result, which assumes Lipschitz continuity of the p-th partial derivatives, generalizes … Read more

    A Bregman alternating direction method of multipliers for sparse probabilistic Boolean network problem

  • Deng Kangkang
  • Peng Zheng
    • Categories Applications - Science and Engineering, Biomedical Applications, Constrained Nonlinear Optimization Tags , , , ,

    The main task of genetic regulatory networks is to construct a sparse probabilistic Boolean network (PBN) based on a given transition-probability matrix and a set of Boolean networks (BNs). In this paper, a Bregman alternating direction method of multipliers (BADMM) is proposed to solve the minimization problem raised in PBN. All the customized subproblem-solvers of … Read more

    A TVSCAD approach for image deblurring with impulsive noise

  • Guoyong Gu
  • Junfeng Yang
  • Suhong Jiang
    • Categories Nonsmooth Optimization Tags , , , , , , , ,

    We consider image deblurring problem in the presence of impulsive noise. It is known that \emph{total variation} (TV) regularization with L1-norm penalized data fitting (TVL1 for short) works reasonably well only when the level of impulsive noise is relatively low. For high level impulsive noise, TVL1 works poorly. The reason is that all data, both … Read more

    On High-order Model Regularization for Constrained Optimization

  • José Mario Martínez
    • Categories Nonlinear Optimization Tags , ,

    In two recent papers regularization methods based on Taylor polynomial models for minimization were proposed that only rely on H\”older conditions on the higher order employed derivatives. Grapiglia and Nesterov considered cubic regularization with a sufficient descent condition that uses the current gradient and resembles the classical Armijo’s criterion. Cartis, Gould, and Toint used Taylor … Read more

    Regularized Stochastic Dual Dynamic Programming for convex nonlinear optimization problems

  • Vincent Guigues
  • Miguel A. Lejeune
  • Wajdi Tekaya
    • Categories Dynamic Programming, Finance and Economics, Stochastic Programming Tags , , , ,

    We define a regularized variant of the Dual Dynamic Programming algorithm called REDDP (REgularized Dual Dynamic Programming) to solve nonlinear dynamic programming equations. We extend the algorithm to solve nonlinear stochastic dynamic programming equations. The corresponding algorithm, called SDDP-REG, can be seen as an extension of a regularization of the Stochastic Dual Dynamic Programming (SDDP) … Read more

    Universal regularization methods – varying the power, the smoothness and the accuracy

  • Coralia Cartis
  • Nicholas I. M. Gould
  • Philippe L. Toint
    • Categories Nonlinear Optimization Tags , ,

    Adaptive cubic regularization methods have emerged as a credible alternative to linesearch and trust-region for smooth nonconvex optimization, with optimal complexity amongst second-order methods. Here we consider a general/new class of adaptive regularization methods, that use first- or higher-order local Taylor models of the objective regularized by a(ny) power of the step size and applied … Read more

    Regularized monotonic regression

  • Oleg Burdakov
  • Oleg Sysoev
    • Categories Quadratic Programming, Statistics Tags , , , , ,

    Monotonic (isotonic) Regression (MR) is a powerful tool used for solving a wide range of important applied problems. One of its features, which poses a limitation on its use in some areas, is that it produces a piecewise constant fitted response. For smoothing the fitted response, we introduce a regularization term in the MR formulated … Read more

    A Sparsity Preserving Convexification Procedure for Indefinite Quadratic Programs Arising in Direct Optimal Control

  • Moritz Diehl
  • Rien Quirynen
  • Robin Verschueren
  • Mario Zanon
    • Categories Systems governed by Differential Equations Optimization Tags , ,

    Quadratic programs (QP) with an indefinite Hessian matrix arise naturally in some direct optimal control methods, e.g. as subproblems in a sequential quadratic programming (SQP) scheme. Typically, the Hessian is approximated with a positive definite matrix to ensure having a unique solution; such a procedure is called \emph{regularization}. We present a novel regularization method tailored … Read more