adaptive regularization

An adaptive regularization algorithm for unconstrained optimization with inexact function and derivatives values

  • Nicholas I. M. Gould
  • Philippe L. Toint
    • Categories Nonlinear Systems and Least-Squares, Unconstrained Optimization Tags , , ,

    An adaptive regularization algorithm for unconstrained nonconvex optimization is proposed that is capable of handling inexact objective-function and derivative values, and also of providing approximate minimizer of arbitrary order. In comparison with a similar algorithm proposed in Cartis, Gould, Toint (2022), its distinguishing feature is that it is based on controlling the relative error between … Read more

    Adaptive Regularization Minimization Algorithms with Non-Smooth Norms

  • Serge Gratton
  • Philippe L. Toint
    • Categories Unconstrained Optimization Tags , , , ,

    A regularization algorithm (AR1pGN) for unconstrained nonlinear minimization is considered, which uses a model consisting of a Taylor expansion of arbitrary degree and regularization term involving a possibly non smooth norm. It is shown that the non-smoothness of the norm does not affect the O(\epsilon_1^{-(p+1)/p}) upper bound on evaluation complexity for finding first-order \epsilon_1-approximate minimizers … Read more

    Hölder Gradient Descent and Adaptive Regularization Methods in Banach Spaces for First-Order Points

  • Serge Gratton
  • Philippe L. Toint
  • Sadok Jerad
    • Categories Unconstrained Optimization Tags , ,

    This paper considers optimization of smooth nonconvex functionals in smooth infinite dimensional spaces. A Hölder gradient descent algorithm is first proposed for finding approximate first-order points of regularized polynomial functionals. This method is then applied to analyze the evaluation complexity of an adaptive regularization method which searches for approximate first-order points of functionals with $\beta$-H\”older … Read more