Nonlinear Optimization

Stochastic Multi-level Composition Optimization Algorithms with Level-Independent Convergence Rates

  • Krishnakumar Balasubramanian
  • Saeed Ghadimi
  • Anthony Nguyen
    • Categories Nonlinear Optimization, Statistics, Stochastic Programming Tags , , ,

    In this paper, we study smooth stochastic multi-level composition optimization problems, where the objective function is a nested composition of $T$ functions. We assume access to noisy evaluations of the functions and their gradients, through a stochastic first-order oracle. For solving this class of problems, we propose two algorithms using moving-average stochastic estimates, and analyze … Read more

    An improved randomized algorithm with noise level tuning for large-scale noisy unconstrained DFO problems

  • Morteza Kimiaei
    • Categories Optimization Software Benchmark, Stochastic Programming, Unconstrained Optimization Tags , , , ,

    In this paper, a new randomized solver (called VRDFON) for noisy unconstrained derivative-free optimization (DFO) problems is discussed. Complexity result in the presence of noise for nonconvex functions is studied. Two effective ingredients of VRDFON are an improved derivative-free line search algorithm with many heuristic enhancements and quadratic models in adaptively determined subspaces. Numerical results … Read more

    On complexity and convergence of high-order coordinate descent algorithms

  • V. S. Amaral
  • Roberto Andreani
  • Ernesto G. Birgin
  • José Mario Martínez
  • Diaulas S. Marcondes
    • Categories Bound-constrained Optimization, Nonlinear Optimization Tags , ,

    Coordinate descent methods with high-order regularized models for box-constrained minimization are introduced. High-order stationarity asymptotic convergence and first-order stationarity worst-case evaluation complexity bounds are established. The computer work that is necessary for obtaining first-order $\varepsilon$-stationarity with respect to the variables of each coordinate-descent block is $O(\varepsilon^{-(p+1)/p})$ whereas the computer work for getting first-order $\varepsilon$-stationarity with … Read more

    Limited-memory Common-directions Method for Large-scale Optimization: Convergence, Parallelization, and Distributed Optimization

  • Ching-pei Lee
  • Po-Wei Wang
  • Chih-Jen Lin
    • Categories Parallel Algorithms, Unconstrained Optimization Tags , , ,

    In this paper, we present a limited-memory common-directions method for smooth optimization that interpolates between first- and second- order methods. At each iteration, a subspace of a limited dimension size is constructed using first-order information from previous iterations, and an ef- ficient Newton method is deployed to find an approximate minimizer within this subspace. With … Read more

    Connecting optimization with spectral analysis of tri-diagonal matrices

  • Jean-Bernard Lasserre
    • Categories Global Optimization, Global Optimization Theory, Nonlinear Optimization Tags

    We show that the global minimum (resp. maximum) of a continuous func- tion on a compact set can be approximated from above (resp. from below) by computing the smallest (rest. largest) eigenvalue of a hierarchy of (r × r) tri-diagonal matrices of increasing size. Equivalently it reduces to computing the smallest (resp. largest) root of … Read more

    Complexity Aspects of Fundamental Questions in Polynomial Optimization

  • Jeffrey Zhang
    • Categories Game Theory, Global Optimization Theory, Nonlinear Optimization Tags , , , ,

    In this thesis, we settle the computational complexity of some fundamental questions in polynomial optimization. These include the questions of (i) finding a local minimum, (ii) testing local minimality of a candidate point, and (iii) deciding attainment of the optimal value. Our results characterize the complexity of these three questions for all degrees of the … Read more

    A Two-level ADMM Algorithm for AC OPF with Convergence Guarantees

  • Xu Andy Sun
  • Kaizhao Sun
    • Categories Constrained Nonlinear Optimization, Network Optimization, Parallel Algorithms Tags , , ,

    This paper proposes a two-level distributed algorithmic framework for solving the AC optimal power flow (OPF) problem with convergence guarantees. The presence of highly nonconvex constraints in OPF poses significant challenges to distributed algorithms based on the alternating direction method of multipliers (ADMM). In particular, convergence is not provably guaranteed for nonconvex network optimization problems … Read more

    Routing and Wavelength Assignment with Protection: A Quadratic Unconstrained Binary Optimization Approach

  • Merve Bodur
  • Hamed Pouya
  • Oylum Şeker
    • Categories 0-1 Programming, Quadratic Programming, Telecommunications

    The routing and wavelength assignment with protection is an important problem in telecommunications. Given an optical network and incoming connection requests, a commonly studied variant of the problem aims to grant maximum number of requests by assigning lightpaths at minimum network resource usage level, while ensuring the provided services remain functional in case of a … Read more

    On scaled stopping criteria for a safeguarded augmented Lagrangian method with theoretical guarantees

  • Roberto Andreani
  • Gabriel Haeser
  • Paulo J. S. Silva
  • María Laura Schuverdt
  • Leonardo D. Secchin
    • Categories Nonlinear Optimization Tags , , ,

    This paper discusses the use of a stopping criterion based on the scaling of the Karush-Kuhn-Tucker (KKT) conditions by the norm of the approximate Lagrange multiplier in the ALGENCAN implementation of a safeguarded augmented Lagrangian method. Such stopping criterion is already used in several nonlinear programming solvers, but it has not yet been considered in … Read more

    A unifying framework for the analysis of projection-free first-order methods under a sufficient slope condition

  • Francesco Rinaldi
  • Damiano Zeffiro
    • Categories Nonlinear Optimization Tags , , , ,

    The analysis of projection-free first order methods is often complicated by the presence of different kinds of “good” and “bad” steps. In this article, we propose a unifying framework for projection-free methods, aiming to simplify the converge analysis by getting rid of such a distinction between steps. The main tool employed in our framework is … Read more