Nonlinear Optimization

Using Filter Methods to Guide Convergence for ADMM, with Applications to Nonnegative Matrix Factorization Problems

  • Robert Baraldi
  • Stefan M. Wild
  • Sven Leyffer
    • Categories Applications - Science and Engineering, Nonlinear Optimization Tags , , ,

    Nonconvex, nonlinear cost functions arise naturally in physical inverse problems and machine learning. The alternating direction method of multipliers (ADMM) has seen extensive use in these applications, despite exhibiting uncertain convergence behavior in many practical nonconvex settings, and struggling with general nonlinear constraints. In contrast, filter methods have proved effective in enforcing convergence for sequential … Read more

    Riemannian Bilevel Optimization

  • Jiaxiang Li
  • Shiqian Ma
    • Categories Nonlinear Optimization Tags ,

    In this work, we consider the bilevel optimization problem on Riemannian manifolds. We inspect the calculation of the hypergradient of such problems on general manifolds and thus enable the utilization of gradient-based algorithms to solve such problems. The calculation of the hypergradient requires utilizing the notion of Riemannian cross-derivative and we inspect the properties and … Read more

    Exploring Nonlinear Distance Metrics for Lipschitz Constant Estimation in Lower Bound Construction for Global Optimization

  • Mohammadsina Almasi
  • Hadis Anahideh
  • Jay Rosenberger
    • Categories Bound-constrained Optimization, Global Optimization, Global Optimization Applications Tags , , , ,

    Bounds play a crucial role in guiding optimization algorithms, improving their speed and quality, and providing optimality gaps. While Lipschitz constant-based lower bound construction is an effective technique, the quality of the linear bounds depends on the function’s topological properties. In this research, we improve upon this by incorporating nonlinear distance metrics and surrogate approximations … Read more

    Refined TSSOS

  • Daria Shaydurova
  • Volker Kaibel
  • Sebastian Sager
    • Categories Global Optimization Applications, Nonlinear Optimization, Semi-definite Programming Tags , , , ,

    The moment-sum of squares hierarchy by Lasserre has become an established technique for solving polynomial optimization problems. It provides a monotonically increasing series of tight bounds, but has well-known scalability limitations. For structured optimization problems, the term-sparsity SOS (TSSOS) approach scales much better due to block-diagonal matrices, obtained by completing the connected components of adjacency … Read more

    Sparse Polynomial Optimization with Unbounded Sets

  • Jie Wang
    • Categories Constrained Nonlinear Optimization, Global Optimization Theory, Semi-definite Programming Tags , , , ,

    This paper considers sparse polynomial optimization with unbounded sets. When the problem possesses correlative sparsity, we propose a sparse homogenized Moment-SOS hierarchy with perturbations to solve it. The new hierarchy introduces one extra auxiliary variable for each variable clique according to the correlative sparsity pattern. Under the running intersection property, we prove that this hierarchy … Read more

    A low-rank augmented Lagrangian method for large-scale semidefinite programming based on a hybrid convex-nonconvex approach

  • Renato D.C. Monteiro
  • Arnesh Sujanani
  • Diego Cifuentes
    • Categories Convex and Nonsmooth Optimization, Linear, Cone and Semidefinite Programming, Nonlinear Optimization Tags , , , , , , ,

    \(\) This paper introduces HALLaR, a new first-order method for solving large-scale semidefinite programs (SDPs) with bounded domain. HALLaR is an inexact augmented Lagrangian (AL) method where the AL subproblems are solved by a novel hybrid low-rank (HLR) method. The recipe behind HLR is based on two key ingredients: 1) an adaptive inexact proximal point … Read more

    Noise-Tolerant Optimization Methods for the Solution of a Robust Design Problem

  • Yuchen Lou
  • Shigeng Sun
  • Jorge Nocedal
    • Categories Nonlinear Optimization, Optimization of Systems modeled by PDEs, Stochastic Programming

    The development of nonlinear optimization algorithms capable of performing reliably in the presence of noise has garnered considerable attention lately. This paper advocates for strategies to create noise-tolerant nonlinear optimization algorithms by adapting classical deterministic methods. These adaptations follow certain design guidelines described here, which make use of estimates of the noise level in the … Read more

    Using generalized simplex methods to approximate derivatives

  • Gabriel Jarry-Bolduc
    • Categories Nonlinear Optimization Tags , ,

    This paper presents two methods for approximating a proper subset of the entries of a Hessian using only function evaluations. These approximations are obtained using the techniques called generalized simplex Hessian and generalized centered simplex Hessian. We show how to choose the matrices of directions involved in the computation of these two techniques depending on … Read more

    The cosine measure relative to a subspace

  • Gabriel Jarry-Bolduc
  • Warren Hare
  • Charles Audet
    • Categories Nonlinear Optimization Tags , , , ,

    The cosine measure was introduced in 2003 to quantify the richness of a finite positive spanning sets of directions in the context of derivative-free directional methods. A positive spanning set is a set of vectors whose nonnegative linear combinations span the whole space. The present work extends the definition of cosine measure. In particular, the … Read more

    Quadratic Optimization Through the Lens of Adjustable Robust Optimization

  • Abbas Khademi
  • Ahmadreza Marandi
    • Categories Quadratic Programming, Robust Optimization Tags , , , ,

    Quadratic optimization (QO) has been studied extensively in the literature due to its applicability in many practical problems. While practical, it is known that QO problems are generally NP-hard. So, researchers developed many approximation methods to find good solutions. In this paper, we go beyond the norm and analyze QO problems using robust optimization techniques. … Read more