worst-case complexity

Complexity of quadratic penalty methods with adaptive accuracy under a PL condition for the constraints

  • Florentin Goyens
  • Geovani Nunes Grapiglia
    • Categories Constrained Nonlinear Optimization Tags , , ,

    We study the quadratic penalty method (QPM) for smooth nonconvex optimization problems with equality constraints. Assuming the constraint violation satisfies the PL condition near the feasible set, we derive sharper worst-case complexity bounds for obtaining approximate first-order KKT points. When the objective and constraints are twice continuously differentiable, we show that QPM equipped with a … Read more

    A Finite-Difference Trust-Region Method for Convexly Constrained Smooth Optimization

  • Dânâ Davar
  • Geovani Nunes Grapiglia
    • Categories Bound-constrained Optimization, Nonlinear Optimization Tags , , , ,

    We propose a derivative-free trust-region method based on finite-difference gradient approximations for smooth optimization problems with convex constraints. The proposed method does not require computing an approximate stationarity measure. For nonconvex problems, we establish a worst-case complexity bound of \(\mathcal{O}\!\left(n\left(\frac{L}{\sigma}\epsilon\right)^{-2}\right)\) function evaluations for the method to reach an \(\left(\frac{L}{\sigma}\epsilon\right)\)-approximate stationary point, where \(n\) is the … Read more

    On the Complexity of Lower-Order Implementations of Higher-Order Methods

  • Nikita Doikov
  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization Tags , , , ,

    In this work, we propose a method for minimizing non-convex functions with Lipschitz continuous \(p\)th-order derivatives, starting from \(p \geq 1\). The method, however, only requires derivative information up to order \((p-1)\), since the \(p\)th-order derivatives are approximated via finite differences. To ensure oracle efficiency, instead of computing finite-difference approximations at every iteration, we reuse … Read more

    A linesearch-based derivative-free method for noisy black-box problems

  • Giampaolo Liuzzi
  • Stefano Lucidi
  • Alberto De Santis
    • Categories Nonlinear Optimization, Stochastic Programming, Unconstrained Optimization Tags , ,

    In this work we consider unconstrained optimization problems. The objective function is known through a zeroth order stochastic oracle that gives an estimate of the true objective function. To solve these problems, we propose a derivativefree algorithm based on extrapolation techniques. Under reasonable assumptions we are able to prove convergence properties for the proposed algorithms. … Read more

    Randomized Subspace Derivative-Free Optimization with Quadratic Models and Second-Order Convergence

  • Lindon Roberts
  • Coralia Cartis
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , ,

    We consider model-based derivative-free optimization (DFO) for large-scale problems, based on iterative minimization in random subspaces. We provide the first worst-case complexity bound for such methods for convergence to approximate second-order critical points, and show that these bounds have significantly improved dimension dependence compared to standard full-space methods, provided low accuracy solutions are desired and/or … Read more

    TRFD: A Derivative-Free Trust-Region Method Based on Finite Differences for Composite Nonsmooth Optimization

  • Dânâ Davar
  • Geovani Nunes Grapiglia
    • Categories Nonsmooth Optimization Tags , , ,

    In this work we present TRFD, a derivative-free trust-region method based on finite differences for minimizing composite functions of the form \(f(x)=h(F(x))\), where \(F\) is a black-box function assumed to have a Lipschitz continuous Jacobian, and \(h\) is a known convex Lipschitz function, possibly nonsmooth. The method approximates the Jacobian of \(F\) via forward finite … Read more

    A Two Stepsize SQP Method for Nonlinear Equality Constrained Stochastic Optimization

  • Michael J. O'Neill
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Stochastic Programming Tags , , , ,

    We develop a Sequential Quadratic Optimization (SQP) algorithm for minimizing a stochastic objective function subject to deterministic equality constraints. The method utilizes two different stepsizes, one which exclusively scales the component of the step corrupted by the variance of the stochastic gradient estimates and a second which scales the entire step. We prove that this … Read more

    Complexity results and active-set identification of a derivative-free method for bound-constrained problems

  • Andrea Brilli
  • Andrea Cristofari
  • Giampaolo Liuzzi
  • Stefano Lucidi
    • Categories Bound-constrained Optimization, Nonlinear Optimization Tags , ,

    In this paper, we analyze a derivative-free line search method designed for bound-constrained problems. Our analysis demonstrates that this method exhibits a worst-case complexity comparable to other derivative-free methods for unconstrained and linearly constrained problems. In particular, when minimizing a function with $n$ variables, we prove that at most ${\cal O(n\epsilon^{-2})}$ iterations are needed to … Read more

    A worst-case complexity analysis for Riemannian non-monotone line-search methods

  • Harry Oviedo
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , ,

    In this paper we deal with non-monotone line-search methods to minimize a smooth cost function on a Riemannian manifold. In particular, we study the number of iterations necessary for this class of algorithms to obtain e-approximated stationary points. Specifically, we prove that under a regularity Lipschitz-type condition on the pullbacks of the cost function to … Read more

    Worst-case evaluation complexity of a derivative-free quadratic regularization method

  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization Tags , , ,

    This short paper presents a derivative-free quadratic regularization method for unconstrained minimization of a smooth function with Lipschitz continuous gradient. At each iteration, trial points are computed by minimizing a quadratic regularization of a local model of the objective function. The models are based on forward finite-difference gradient approximations. By using a suitable acceptance condition … Read more