Nonlinear Optimization

Generalized asymmetric forward-backward-adjoint algorithms for convex-concave saddle-point problem

  • Jianchao Bai
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , ,

    The convex-concave minimax problem, also known as the saddle-point problem, has been extensively studied from various aspects including the algorithm design, convergence condition and complexity. In this paper, we propose a generalized asymmetric forward-backward-adjoint algorithm (G-AFBA) to solve such a problem by utilizing both the proximal techniques and the extrapolation of primal-dual updates. Besides applying … Read more

    Limited memory gradient methods for unconstrained optimization

  • Giulia Ferrandi
  • Michiel E. Hochstenbach
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , , ,

    The limited memory steepest descent method (Fletcher, 2012) for unconstrained optimization problems stores a few past gradients to compute multiple stepsizes at once. We review this method and propose new variants. For strictly convex quadratic objective functions, we study the numerical behavior of different techniques to compute new stepsizes. In particular, we introduce a method … Read more

    Adaptive Consensus: A network pruning approach for decentralized optimization

  • Suhail Mohmad Shah
  • Albert S. Berahas
  • Raghu Bollapragada
    • Categories Convex Optimization, Network Optimization, Nonlinear Optimization Tags , ,

    We consider network-based decentralized optimization problems, where each node in the network possesses a local function and the objective is to collectively attain a consensus solution that minimizes the sum of all the local functions. A major challenge in decentralized optimization is the reliance on communication which remains a considerable bottleneck in many applications. To … Read more

    Inexact Newton methods with matrix approximation by sampling for nonlinear least-squares and systems

  • Stefania Bellavia
  • Greta Malaspina
  • Benedetta Morini
    • Categories Nonlinear Optimization, Nonlinear Systems and Least-Squares, Stochastic Programming

    We develop and analyze stochastic inexact Gauss-Newton methods for nonlinear least-squares problems and inexact Newton methods for nonlinear systems of equations. Random models are formed using suitable sampling strategies for the matrices involved in the deterministic models. The analysis of the expected number of iterations needed in the worst case to achieve a desired level … Read more

    Sufficient Conditions for Lipschitzian Error Bounds for Complementarity Systems

  • Mario Jelitte
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , , , , ,

    We are concerned with Lipschitzian error bounds and Lipschitzian stability properties for solutions of a complementarity system. For this purpose, we deal with a nonsmooth slack-variable reformulation of the complementarity system, and study conditions under which the reformulation serves as a local error bound for the solution set of the complementarity system. We also discuss … Read more

    Expected decrease for derivative-free algorithms using random subspaces

  • Lindon Roberts
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags ,

    Derivative-free algorithms seek the minimum of a given function based only on function values queried at appropriate points. Although these methods are widely used in practice, their performance is known to worsen as the problem dimension increases. Recent advances in developing randomized derivative-free techniques have tackled this issue by working in low-dimensional subspaces that are … Read more

    Almost-sure convergence of iterates and multipliers in stochastic sequential quadratic optimization

  • Frank E. Curtis
  • Xin Jiang
  • Qi Wang
    • Categories Constrained Nonlinear Optimization, Data Science Applications, Stochastic Programming Tags , , ,

    Stochastic sequential quadratic optimization (SQP) methods for solving continuous optimization problems with nonlinear equality constraints have attracted attention recently, such as for solving large-scale data-fitting problems subject to nonconvex constraints. However, for a recently proposed subclass of such methods that is built on the popular stochastic-gradient methodology from the unconstrained setting, convergence guarantees have been … Read more

    Transformation of Bilevel Optimization Problems into Single-Level Ones

  • Stephan Dempe
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Other Topics Tags

    Bilevel optimization problems are hierarchical problems with a constraint set which is a subset of the graph of the solution set mapping of a second optimization problem. To investigate their properties and derive solution algorithms, their transformation into single-level ones is necessary. For this, various approaches have been developed. The rst and most often used … Read more

    Inexact Direct-Search Methods for Bilevel Optimization Problems

  • Youssef Diouane
  • Vyacheslav Kungurtsev
  • Francesco Rinaldi
  • Damiano Zeffiro
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Optimization in Data Science

    In this work, we introduce new direct search schemes for the solution of bilevel optimization (BO) problems. Our methods rely on a fixed accuracy black box oracle for the lower-level problem, and deal both with smooth and potentially nonsmooth true objectives. We thus analyze for the first time in the literature direct search schemes in … Read more