Stefan M. Wild When derivatives of a nonlinear objective function are unavailable, many derivative- free optimization algorithms rely on interpolation-based models of the function. But what if the function values are contaminated by noise, as in most of the simulation- based problems typically encountered in this area? We propose to obtain linear and quadratic models by using knowledge … Read more
We analyze the relationship between the noise level of a function and the accuracy and reliability of derivatives and difference estimates. We derive and empirically validate measures of quality for both derivatives and difference estimates. Using these measures, we quantify the accuracy of derivatives and differences in terms of the noise level of the function. … Read more
We analyze globally convergent derivative-free trust region algorithms relying on radial basis function interpolation models. Our results extend the recent work of Conn, Scheinberg, and Vicente to fully linear models that have a nonlinear term. We characterize the types of radial basis functions that fit in our analysis and thus show global convergence to first-order … Read more
We employ recent work on computational noise to obtain near-optimal finite difference estimates of the derivatives of a noisy function. Our analysis employs a stochastic model of the noise without assuming a specific form of distribution. We use this model to derive theoretical bounds for the errors in the difference estimates and obtain an easily … Read more
Computational noise in deterministic simulations is as ill-defined a concept as can be found in scientific computing. When coupled with adaptive strategies, the effects of finite precision destroy smoothness of the simulation output and complicate subsequent analysis. Following the work of Hamming on roundoff errors, we present a new algorithm, ECnoise, for quantifying the noise … Read more
We present a new derivative-free algorithm, ORBIT, for unconstrained local optimization of computationally expensive functions. A trust-region framework using interpolating Radial Basis Function (RBF) models is employed. The RBF models considered often allow ORBIT to interpolate nonlinear functions using fewer function evaluations than the polynomial models considered by present techniques. Approximation guarantees are obtained by … Read more
We propose data profiles as a tool for analyzing the performance of derivative-free optimization solvers when there are constraints on the computational budget. We use performance and data profiles, together with a convergence test that measures the decrease in function value, to analyze the performance of three solvers on sets of smooth, noisy, and piecewise-smooth … Read more