derivative-free optimization

Direct search based on probabilistic feasible descent for bound and linearly constrained problems

  • Serge Gratton
  • Clément W. Royer
  • Luis Nunes Vicente
  • Z. Zhang
    • Categories Bound-constrained Optimization, Nonlinear Optimization, Stochastic Programming Tags , , , , ,

    Direct search is a methodology for derivative-free optimization whose iterations are characterized by evaluating the objective function using a set of polling directions. In deterministic direct search applied to smooth objectives, these directions must somehow conform to the geometry of the feasible region and typically consist of positive generators of approximate tangent cones (which then … Read more

    Locally weighted regression models for surrogate-assisted design optimization

  • Charles Audet
  • Michael Kokkolaras
  • Sébastien Le Digabel
  • Bastien Talgorn
    • Categories Mechanical Engineering, Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    Locally weighted regression combines the advantages of polynomial regression and kernel smoothing. We present three ideas for appropriate and effective use of LOcally WEighted Scatterplot Smoothing (LOWESS) models for surrogate optimization. First, a method is proposed to reduce the computational cost of LOWESS models. Second, a local scaling coefficient is introduced to adapt LOWESS models … Read more

    A progressive barrier derivative-free trust-region algorithm for constrained optimization

  • Charles Audet
  • Andrew R. Conn
  • Sébastien Le Digabel
  • Mathilde Peyrega
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , ,

    We study derivative-free constrained optimization problems and propose a trust-region method that builds linear or quadratic models around the best feasible and and around the best infeasible solutions found so far. These models are optimized within a trust region, and the progressive barrier methodology handles the constraints by progressively pushing the infeasible solutions toward the … Read more

    Manifold Sampling for L1 Nonconvex Optimization

  • Jeffrey Larson
  • Matt Menickelly
  • Stefan M. Wild
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , ,

    We present a new algorithm, called manifold sampling, for the unconstrained minimization of a nonsmooth composite function $h\circ F$ when $h$ has known structure. In particular, by classifying points in the domain of the nonsmooth function $h$ into manifolds, we adapt search directions within a trust-region framework based on knowledge of manifolds intersecting the current … Read more

    Global convergence of a derivative-free inexact restoration filter algorithm for nonlinear programming

  • Priscila Savulski Ferreira
  • Elizabeth Wegner Karas
  • Mael Sachine
  • Francisco N. C. Sobral
    • Categories Constrained Nonlinear Optimization Tags , , , , ,

    In this work we present an algorithm for solving constrained optimization problems that does not make explicit use of the objective function derivatives. The algorithm mixes an inexact restoration framework with filter techniques, where the forbidden regions can be given by the flat or slanting filter rule. Each iteration is decomposed in two independent phases: … Read more

    Randomized Derivative-Free Optimization of Noisy Convex Functions

  • Ruobing Chen
  • Stefan M. Wild
    • Categories Stochastic Programming, Unconstrained Optimization Tags , , ,

    We propose STARS, a randomized derivative-free algorithm for unconstrained optimization when the function evaluations are contaminated with random noise. STARS takes dynamic, noise-adjusted smoothing step-sizes that minimize the least-squares error between the true directional derivative of a noisy function and its finite difference approximation. We provide a convergence rate analysis of STARS for solving convex … Read more

    BFO, a trainable derivative-free Brute Force Optimizer for nonlinear bound-constrained optimization and equilibrium computations with continuous and discrete variables

  • Margherita Porcelli
  • Philippe L. Toint
    • Categories (Mixed) Integer Nonlinear Programming, Bound-constrained Optimization, Nonlinear Optimization Tags , , , ,

    A direct-search derivative-free Matlab optimizer for bound-constrained problems is described, whose remarkable features are its ability to handle a mix of continuous and discrete variables, a versatile interface as well as a novel self-training option. Its performance compares favourably with that of NOMAD, a state-of-the art package. It is also applicable to multilevel equilibrium- or … Read more

    A second-order globally convergent direct-search method and its worst-case complexity

  • Serge Gratton
  • Clément W. Royer
  • Luis Nunes Vicente
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    Direct-search algorithms form one of the main classes of algorithms for smooth unconstrained derivative-free optimization, due to their simplicity and their well-established convergence results. They proceed by iteratively looking for improvement along some vectors or directions. In the presence of smoothness, first-order global convergence comes from the ability of the vectors to approximate the steepest … Read more

    Stochastic Optimization using a Trust-Region Method and Random Models

  • Ruobing Chen
  • Matt Menickelly
  • Katya Scheinberg
    • Categories Stochastic Programming, Unconstrained Optimization Tags , , ,

    In this paper, we propose and analyze a trust-region model-based algorithm for solving unconstrained stochastic optimization problems. Our framework utilizes random models of an objective function $f(x)$, obtained from stochastic observations of the function or its gradient. Our method also utilizes estimates of function values to gauge progress that is being made. The convergence analysis … Read more

    A trust-funnel method for nonlinear optimization problems with general nonlinear constraints and its application to derivative-free optimization

  • Phillipe Sampaio
  • Philippe L. Toint
    • Categories Constrained Nonlinear Optimization Tags , , ,

    A trust-funnel method is proposed for solving nonlinear optimization problems with general nonlinear constraints. It extends the one presented by Gould and Toint (Math. Prog., 122(1):155-196, 2010), originally proposed for equality-constrained optimization problems only, to problems with both equality and inequality constraints and where simple bounds are also considered. As the original one, our method … Read more