Nonlinear Optimization

Inexact Sequential Quadratic Optimization for Minimizing a Stochastic Objective Function Subject to Deterministic Nonlinear Equality Constraints

  • Frank E. Curtis
  • Daniel P. Robinson
  • Baoyu Zhou
    • Categories Constrained Nonlinear Optimization, Stochastic Programming Tags , , , ,

    An algorithm is proposed, analyzed, and tested experimentally for solving stochastic optimization problems in which the decision variables are constrained to satisfy equations defined by deterministic, smooth, and nonlinear functions. It is assumed that constraint function and derivative values can be computed, but that only stochastic approximations are available for the objective function and its … Read more

    A study of Liu-Storey conjugate gradient methods for vector optimization

  • Max L. N. Gonçalves
  • Leandro Fonseca Prudente
  • F. S. Lima
    • Categories Multi-Criteria Optimization, Unconstrained Optimization Tags , , ,

    This work presents a study of Liu-Storey (LS) nonlinear conjugate gradient (CG) methods to solve vector optimization problems. Three variants of the LS-CG method originally designed to solve single-objective problems are extended to the vector setting. The first algorithm restricts the LS conjugate parameter to be nonnegative and use a sufficiently accurate line search satisfying … Read more

    A framework for convex-constrained monotone nonlinear equations and its special cases

  • Max L. N. Gonçalves
  • Tiago Menezes
    • Categories Nonlinear Systems and Least-Squares

    This work refers to methods for solving convex-constrained monotone nonlinear equations. We first propose a framework, which is obtained by combining a safeguard strategy on the search directions with a notion of approximate projections. The global convergence of the framework is established under appropriate assumptions and some examples of methods which fall into this framework … Read more

    An Efficient Retraction Mapping for the Symplectic Stiefel Manifold

  • Rafael Herrera
  • Harry Oviedo
    • Categories Constrained Nonlinear Optimization Tags , , ,

    This article introduces a new retraction on the symplectic Stiefel manifold. The operation that requires the highest computational cost to compute the novel retraction is a matrix inversion of size $2p$–by–$2p$, which is much less expensive than those required for the available retractions in the literature. Later, with the new retraction, we design a constraint … Read more

    A Fixed Point Approach with a New Solution Concept for Set-valued Optimization

  • Shengjie Li
  • Christiane Tammer
  • Chaoli Yao
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization

    We present a fixed point approach to find the whole solution set of a set-valued optimization problem though a parametric problem, in which the height of the level set of the objective function is regarded as the parameter. First, the solution concept based on the vector approach is considered in this method. Then, we propose … Read more

    A Stochastic Sequential Quadratic Optimization Algorithm for Nonlinear Equality Constrained Optimization with Rank-Deficient Jacobians

  • Albert S. Berahas
  • Frank E. Curtis
  • Michael J. O'Neill
  • Daniel P. Robinson
    • Categories Constrained Nonlinear Optimization, Statistics Tags , , , ,

    A sequential quadratic optimization algorithm is proposed for solving smooth nonlinear equality constrained optimization problems in which the objective function is defined by an expectation of a stochastic function. The algorithmic structure of the proposed method is based on a step decomposition strategy that is known in the literature to be widely effective in practice, … Read more

    A stochastic first-order trust-region method with inexact restoration for finite-sum minimization

  • Stefania Bellavia
  • Nataša Krejić
  • Benedetta Morini
  • Simone Rebegoldi
    • Categories Nonlinear Optimization Tags , , , ,

    We propose a stochastic first-order trust-region method with inexact function and gradient evaluations for solving finite-sum minimization problems. At each iteration, the function and the gradient are approximated by sampling. The sample size in gradient approximations is smaller than the sample size in function approximations and the latter is determined using a deterministic rule inspired … Read more

    Frank-Wolfe and friends: a journey into projection-free first-order optimization methods

  • Immanuel M. Bomze
  • Francesco Rinaldi
  • Damiano Zeffiro
    • Categories Convex Optimization, Nonlinear Optimization Tags , , , ,

    Invented some 65 years ago in a seminal paper by Marguerite Straus-Frank and Philip Wolfe, the Frank-Wolfe method recently enjoys a remarkable revival, fuelled by the need of fast and reliable first-order optimization methods in Data Science and other relevant application areas. This review tries to explain the success of this approach by illustrating versatility … Read more

    Cardinality Minimization, Constraints, and Regularization: A Survey

  • Daniel Bienstock
  • Andrea Lodi
  • Andreas M. Tillmann
  • Alexandra Schwartz
    • Categories (Mixed) Integer Linear Programming, (Mixed) Integer Nonlinear Programming, Constrained Nonlinear Optimization Tags , , , , , , ,

    We survey optimization problems that involve the cardinality of variable vectors in constraints or the objective function. We provide a unified viewpoint on the general problem classes and models, and give concrete examples from diverse application fields such as signal and image processing, portfolio selection, or machine learning. The paper discusses general-purpose modeling techniques and … Read more

    LSOS: Line-search Second-Order Stochastic optimization methods for nonconvex finite sums

  • Daniela di Serafino
  • Nataša Krejić
  • Nataša Krklec Jerinkić
  • Marco Viola
    • Categories Nonlinear Optimization, Stochastic Programming Tags , , , ,

    We develop a line-search second-order algorithmic framework for minimizing finite sums. We do not make any convexity assumptions, but require the terms of the sum to be continuously differentiable and have Lipschitz-continuous gradients. The methods fitting into this framework combine line searches and suitably decaying step lengths. A key issue is a two-step sampling at … Read more