Clément W. Royer – Page 2 – Optimization Online

A decoupled first/second-order steps technique for nonconvex nonlinear unconstrained optimization with improved complexity bounds

Published: 2017/04/21, Updated: 2018/02/21

Nonlinear Optimization, Unconstrained Optimization decoupling, first-order stationarity, global rates, second-order stationarity, trust-region methods, worst-case complexity

In order to be provably convergent towards a second-order stationary point, optimization methods applied to nonconvex problems must necessarily exploit both first and second-order information. However, as revealed by recent complexity analyzes of some of these methods, the overall effort to reach second-order points is significantly larger when compared to the one of approaching first-order … Read more

Complexity and global rates of trust-region methods based on probabilistic models

Published: 2017/02/17

Stochastic Programming, Unconstrained Optimization global rates, probabilistic models, trust-region methods, worst-case complexity

Trust-region algorithms have been proved to globally converge with probability one when the accuracy of the trust-region models is imposed with a certain probability conditioning on the iteration history. In this paper, we study their complexity, providing global rates and worst case complexity bounds on the number of iterations (with overwhelmingly high probability), for both … Read more

Direct search based on probabilistic feasible descent for bound and linearly constrained problems

Published: 2017/02/17

Bound-constrained Optimization, Nonlinear Optimization, Stochastic Programming bounds, derivative-free optimization, direct search, linear constraints, positive generators, probabilistic feasible descent

Direct search is a methodology for derivative-free optimization whose iterations are characterized by evaluating the objective function using a set of polling directions. In deterministic direct search applied to smooth objectives, these directions must somehow conform to the geometry of the feasible region and typically consist of positive generators of approximate tangent cones (which then … Read more

A second-order globally convergent direct-search method and its worst-case complexity

Published: 2015/06/04

Serge Gratton

Clément W. Royer

Luis Nunes Vicente

Nonlinear Optimization, Unconstrained Optimization derivative-free optimization, direct search, second-order, worst-case complexity

Direct-search algorithms form one of the main classes of algorithms for smooth unconstrained derivative-free optimization, due to their simplicity and their well-established convergence results. They proceed by iteratively looking for improvement along some vectors or directions. In the presence of smoothness, first-order global convergence comes from the ability of the vectors to approximate the steepest … Read more

Direct search based on probabilistic descent

Published: 2014/03/19

Convex and Nonsmooth Optimization, Nonlinear Optimization, Stochastic Programming derivative-free optimization, direct search, polling, positive spanning sets, probabilistic descent, random directions

Direct-search methods are a class of popular derivative-free algorithms characterized by evaluating the objective function using a step size and a number of (polling) directions. When applied to the minimization of smooth functions, the polling directions are typically taken from positive spanning sets which in turn must have at least n+1 vectors in an n-dimensional … Read more