convergence bounds

Parametric complexity analysis for a class of first-order Adagrad-like algorithms

  • Serge Gratton
  • Sadok Jerad
  • Philippe L. Toint
    • Categories Stochastic Programming, Unconstrained Optimization Tags , , , , ,

    A class of algorithms for optimization in the presence of noise is presented, that does not require the evaluation of the objective function. This class generalizes the well-known Adagrad method. The complexity of this class is then analyzed as a function of its parameters, and it is shown that some methods of the class enjoy … Read more

    First-Order Objective-Function-Free Optimization Algorithms and Their Complexity

  • Serge Gratton
  • Sadok Jerad
  • Philippe L. Toint
    • Categories Unconstrained Optimization Tags , , , , ,

    A class of algorithms for unconstrained nonconvex optimization is considered where the value of the objective function is never computed. The class contains a deterministic version of the first-order Adagrad method typically used for minimization of noisy function, but also allows the use of second-order information when available. The rate of convergence of methods in … Read more