finite differences

Quadratic Regularization Methods with Finite-Difference Gradient Approximations

  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization Tags , , ,

    This paper presents two quadratic regularization methods with finite-difference gradient approximations for smooth unconstrained optimization problems. One method is based on forward finite-difference gradients, while the other is based on central finite-difference gradients. In both methods, the accuracy of the gradient approximations and the regularization parameter in the quadratic models are jointly adjusted using a … Read more

    Adaptive Finite-Difference Interval Estimation for Noisy Derivative-Free Optimization

  • Hao-Jun Shi
  • Yuchen Xie
  • Melody Xuan
  • Jorge Nocedal
    • Categories Nonlinear Optimization Tags , , , ,

    A common approach for minimizing a smooth nonlinear function is to employ finite-difference approximations to the gradient. While this can be easily performed when no error is present within the function evaluations, when the function is noisy, the optimal choice requires information about the noise level and higher-order derivatives of the function, which is often … Read more

    On the Numerical Performance of Derivative-Free Optimization Methods Based on Finite-Difference Approximations

  • Jorge Nocedal
  • Hao-Jun Shi
  • Melody Xuan
  • Figen Oztoprak
    • Categories Nonlinear Optimization Tags , , , ,

    The goal of this paper is to investigate an approach for derivative-free optimization that has not received sufficient attention in the literature and is yet one of the simplest to implement and parallelize. It consists of computing gradients of a smoothed approximation of the objective function (and constraints), and employing them within established codes. These … Read more

    A Theoretical and Empirical Comparison of Gradient Approximations in Derivative-Free Optimization

  • Albert S. Berahas
  • Liyuan Cao
  • Katya Scheinberg
  • Krzysztof Choromanski
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    In this paper, we analyze several methods for approximating gradients of noisy functions using only function values. These methods include finite differences, linear interpolation, Gaussian smoothing and smoothing on a unit sphere. The methods differ in the number of functions sampled, the choice of the sample points, and the way in which the gradient approximations … Read more

    On the oracle complexity of first-order and derivative-free algorithms for smooth nonconvex minimization

  • Coralia Cartis
  • Nicholas I. M. Gould
  • Philippe L. Toint
    • Categories Nonlinear Systems and Least-Squares, Unconstrained Optimization Tags , , , , ,

    The (optimal) function/gradient evaluations worst-case complexity analysis available for the Adaptive Regularizations algorithms with Cubics (ARC) for nonconvex smooth unconstrained optimization is extended to finite-difference versions of this algorithm, yielding complexity bounds for first-order and derivative free methods applied on the same problem class. A comparison with the results obtained for derivative-free methods by Vicente … Read more

    WASP: a Wavelet Adaptive Solver for boundary value Problems – Short Reference Manual

  • Jean-Baptiste Caillau
  • Joseph Noailles
    • Categories Control Applications Tags , , , ,

    This is a short guide to use the Matlab package WASP designed for the numerical solution of two-point linear boundary value problems that arise typically in linear quadratic optimal control. The method relies upon an adaptive computation of discretization based on a wavelet analysis. On a given refined grid, finite differences of various order are … Read more