Nonlinear Optimization

Lower Bounds for Feasibility and Stationarity in First-Order Nonconvex Constrained Optimization

  • Qiankun Shi
  • Xiao Wang
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    We study oracle-complexity lower bounds for first-order methods applied to smooth equality-constrained nonconvex optimization, separating two components of approximate KKT accuracy: feasibility and stationarity. Under iteration-wise Jacobian regularity, we prove a lower bound of order \(\Omega(\frac{L_c\Delta_c}{\sigma^2}+\log\log(\frac{\sigma^2}{L_c\epsilon}))\) to achieve \(\epsilon\)-feasibility. For stationarity, we construct a nonlinear equality-constrained hard instance whose multiplier-minimized stationarity residual reduces exactly to … Read more

    A Fletcher’s Augmented Lagrangian-Based Stochastic First-Order Method for Nonconvex Equality-Constrained Optimization

  • Yawen Cui
  • Qiankun Shi
  • Xiao Wang
  • Xiantao Xiao
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    In this paper, we study nonconvex equality-constrained optimization problems in which only stochastic first-order approximations of the objective and constraint functions are available. Owing to the stochasticity in both objective and constraints, most existing stochastic first-order methods incur relatively high oracle complexity, particularly in terms of stochastic constraint function evaluations. To address this issue, we … Read more

    Constrained Variable Projection for Structured Problems

  • Francesco Rinaldi
    • Categories Bilevel Optimization, Nonlinear Optimization, Optimization in Data Science Tags , ,

    Variable projection is a classical technique for separable nonlinear least-squares problems, in which variables that enter linearly are eliminated exactly, yielding a reduced nonlinear problem. By expressing this framework as a particular instance of a broader class of bilevel optimization problems, we develop a constrained variable-projection framework for data-science models, where the remaining variables are … Read more

    Local-to-Global Exactness of SDP Relaxations for Sparse QCQPs

  • Masakazu Kojima
  • Sunyoung Kim
  • Naohiko Arima
    • Categories Global Optimization Theory, Quadratic Programming, Semi-definite Programming Tags , , , , , , , ,

    We study exact semidefinite programming (SDP) relaxation for a given sparse quadratically constrained quadratic program (QCQP). The SDP relaxation is exact if, whenever it has an optimal solution, it admits a rank-at-most-one optimal solution that corresponds to an optimal solution of the QCQP. Using the maximal cliques of a chordal extension of the aggregate sparsity … Read more

    Quadratic Quasi-Newton Optimization – An Interpolated Hybrid Method

  • Andrew Charneski
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags ,

    We present the Quadratic-Quasi-Newton (QQN) algorithm, a novel optimization method that combines gradient descent and quasi-Newton directions through quadratic interpolation. QQN constructs a parametric path d(t) = t(1 − t)(−∇f) + t 2 d L-BFGS and performs univariate optimization along this path, creating an adaptive interpolation that requires no additional hyperparameters beyond those of its … Read more

    D-optimal partitioning: design of experiments under heterogeneous treatment effects

  • Jiaqi Wang
  • Weijun Xie
    • Categories Combinatorial Optimization, Nonlinear Optimization, Optimization in Data Science Tags , , , ,

    Modern experimentation in business and public policy often studies targeted interventions whose effects depend on the heterogeneous attributes of individuals. We examine heterogeneous treatment effects through the lens of optimal design of experiments, which allocates treatment decisions to maximize the precision of estimated treatment-covariate interactions. We introduce the D-optimal partitioning problem for balancing the information … Read more

    PyROS: The Pyomo Robust Optimization Solver

  • Jason A. F. Sherman
  • Natalie M. Isenberg
  • John D. Siirola
  • Chrysanthos E. Gounaris
    • Categories Nonlinear Optimization, Optimization Software and Modeling Systems, Robust Optimization Tags , , , ,

    We present PyROS, a Python-based meta-solver that automates a generalized cutting-set algorithm for the solution of nonconvex two-stage robust optimization (RO) problems with uncertain equality constraints. Freely available through the open-source optimization software package Pyomo, PyROS is designed to operate on a user-provided deterministic model and uncertainty set, such that a solution to the RO … Read more

    Distributionally Robust Optimization with General Uncertainty Structure

  • Judith Brugman
  • Dimitris Bertsimas
  • Johan Van Leeuwaarden
  • Dick den Hertog
    • Categories Nonlinear Optimization, Robust Optimization, Stochastic Programming Tags , , , , , ,

    We develop an exact solution framework for a broad class of Distributionally Robust Optimization (DRO) problems with general uncertainty structure. Within the class of moment- and confidence-set-based ambiguity sets, existing exact methods are largely limited to max-of-affine functions under ambiguity sets with strictly nested confidence sets. To enlarge this scope while preserving tractability, we introduce … Read more

    On the existence of Lagrange multipliers in conic programming

  • Gabriel Haeser
  • Daiana Santos
    • Categories Constrained Nonlinear Optimization Tags ,

    The existence of Lagrange multipliers at a solution of a nonlinear optimization problem constitutes one of the cornerstones of modern optimization theory, with many important consequences for guiding algorithmic procedures towards a solution, defining stopping criteria, performing stability analysis, and several other aspects. However, the proof of this result is often intricate, relying on non-trivial … Read more

    Inexactly Smooth Performance Estimation and New Optimized Gradient Methods

  • Aaron Zoll
  • Benjamin Grimmer
    • Categories Convex Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

      We consider a general class of “inexactly smooth” convex functions, providing a universal model capturing as special cases $L$-smooth, $M$-Lipschitz, and H\”older smooth functions, and any combination thereof. Such functions possess a calculus closely following that of smooth functions. Our main results provide inexactly smooth functions with interpolation theorems that are necessary and sufficient … Read more