high-order methods

On the Complexity of Lower-Order Implementations of Higher-Order Methods

  • Nikita Doikov
  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization Tags , , , ,

    In this work, we propose a method for minimizing non-convex functions with Lipschitz continuous \(p\)th-order derivatives, starting from \(p \geq 1\). The method, however, only requires derivative information up to order \((p-1)\), since the \(p\)th-order derivatives are approximated via finite differences. To ensure oracle efficiency, instead of computing finite-difference approximations at every iteration, we reuse … Read more

    Worst-Case Complexity of High-Order Algorithms for Pareto-Front Reconstruction

  • Andrea Cristofari
  • Marianna De Santis
  • Giampaolo Liuzzi
  • Stefano Lucidi
    • Categories Multi-Criteria Optimization, Nonlinear Optimization Tags , ,

    In this paper, we are concerned with a worst-case complexity analysis of a-posteriori algorithms for unconstrained multiobjective optimization. Specifically, we propose an algorithmic framework that generates sets of points by means of $p$th-order models regularized with a power $p+1$ of the norm of the step. Through a tailored search procedure, several trial points are generated … Read more

    On Inexact Solution of Auxiliary Problems in Tensor Methods for Convex Optimization

  • Geovani Nunes Grapiglia
  • Yurii Nesterov
    • Categories Convex and Nonsmooth Optimization Tags , , , ,

    In this paper we study the auxiliary problems that appear in p-order tensor methods for unconstrained minimization of convex functions with \nu-Holder continuous pth derivatives. This type of auxiliary problems corresponds to the minimization of a (p+\nu)-order regularization of the pth order Taylor approximation of the objective. For the case p=3, we consider the use … Read more

    Tensor Methods for Finding Approximate Stationary Points of Convex Functions

  • Geovani Nunes Grapiglia
  • Yurii Nesterov
    • Categories Convex Optimization Tags , , , ,

    In this paper we consider the problem of finding \epsilon-approximate stationary points of convex functions that are p-times differentiable with \nu-Hölder continuous pth derivatives. We present tensor methods with and without acceleration. Specifically, we show that the non-accelerated schemes take at most O(\epsilon^{-1/(p+\nu-1)}) iterations to reduce the norm of the gradient of the objective below … Read more

    Tensor Methods for Minimizing Convex Functions with Hölder Continuous Higher-Order Derivatives

  • Geovani Nunes Grapiglia
  • Yurii Nesterov
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , ,

    In this paper we study p-order methods for unconstrained minimization of convex functions that are p-times differentiable with $\nu$-Hölder continuous pth derivatives. We propose tensor schemes with and without acceleration. For the schemes without acceleration, we establish iteration complexity bounds of $\mathcal{O}\left(\epsilon^{-1/(p+\nu-1)}\right)$ for reducing the functional residual below a given $\epsilon\in (0,1)$. Assuming that $\nu$ … Read more