Nonlinear Optimization

Doubly stochastic primal dual splitting algorithm with variance reduction for saddle point problems

  • Dimitri Papadimitriou
    • Categories Constrained Nonlinear Optimization, Convex Optimization, Nonlinear Optimization, Unconstrained Optimization Tags , , , , ,

    The structured saddle-point problem involving the infimal convolution in real Hilbert spaces finds applicability in many applied mathematics disciplines. For this purpose, we develop a stochastic primal-dual splitting algorithm with loopless variance-reduction for solving this generic problem. We first prove the weak almost sure convergence of the iterates. We then demonstrate that our algorithm achieves … Read more

    Singular value half thresholding algorithm for lp regularized matrix optimization problems

  • Peng Dingtao
    • Categories Nonsmooth Optimization, Unconstrained Optimization Tags , , , ,

    In this paper, we study the low-rank matrix optimization problem, where the penalty term is the $\ell_p~(0<p<1)$ regularization. Inspired by the good performance of half thresholding function in sparse/low-rank recovery problems, we propose a singular value half thresholding (SVHT) algorithm to solve the $\ell_p$ regularized matrix optimization problem. The main iteration in SVHT algorithm makes … Read more

    Near-optimal closed-loop method via Lyapunov damping for convex optimization

  • Camille Castera
  • Peter Ochs
    • Categories Convex Optimization, Systems governed by Differential Equations Optimization, Unconstrained Optimization Tags , , ,

    We introduce an autonomous system with closed-loop damping for first-order convex optimization. While, to this day, optimal rates of convergence are only achieved by non-autonomous methods via open-loop damping (e.g., Nesterov’s algorithm), we show that our system is the first one featuring a closed-loop damping while exhibiting a rate arbitrarily close to the optimal one. … Read more

    An outer approximation method for solving mixed-integer convex quadratic programs with indicators

  • Linchuan Wei
  • Simge Küçükyavuz
    • Categories (Mixed) Integer Nonlinear Programming, Quadratic Programming Tags , ,

    Mixed-integer convex quadratic programs with indicator variables (MIQP) encompass a wide range of applications, from statistical learning to energy, finance, and logistics. The outer approximation (OA) algorithm has been proven efficient in solving MIQP, and the key to the success of an OA algorithm is the strength of the cutting planes employed. In this paper, … Read more

    Local Convergence Analysis of an Inexact Trust-Region Method for Nonsmooth Optimization

  • Robert Baraldi
  • Drew P. Kouri
    • Categories Infinite Dimensional Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , , , ,

    In [R. J. Baraldi and D. P. Kouri, Mathematical Programming, (2022), pp. 1–40], we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex function and a nonsmooth convex function in Hilbert space—a class of problems that is ubiquitous in data science, learning, optimal control, and inverse problems. This algorithm has demonstrated … Read more

    Efficient Proximal Subproblem Solvers for a Nonsmooth Trust-Region Method

  • Robert Baraldi
  • Drew P. Kouri
    • Categories Infinite Dimensional Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    In [R. J. Baraldi and D. P. Kouri, Mathematical Programming, (2022), pp. 1-40], we introduced an inexact trust-region algorithm for minimizing the sum of a smooth nonconvex and nonsmooth convex function. The principle expense of this method is in computing a trial iterate that satisfies the so-called fraction of Cauchy decrease condition—a bound that ensures … Read more

    Budget-Constrained Maximization of “Cobb-Douglas with Linear Components” Utility Function

  • Somdeb Lahiri
    • Categories Constrained Nonlinear Optimization, Linear Programming, Other Topics Tags , ,

    In what follows, we provide the demand analysis associated with budget-constrained linear utility maximization for each of several categories of goods, with the marginal rate of consumption expenditure-as a share of wealth- being a positive constant less than or equal to one. The marginal rate of consumption expenditure is endogenously determined, by a budget-constrained “Cobb-Douglas … Read more

    The limitation of neural nets for approximation and optimization

  • Tommaso Giovannelli
  • Oumaima Sohab
  • Luis Nunes Vicente
    • Categories Global Optimization, Nonlinear Optimization, Optimization in Data Science Tags , , ,

    We are interested in assessing the use of neural networks as surrogate models to approximate and minimize objective functions in optimization problems. While neural networks are widely used for machine learning tasks such as classification and regression, their application in solving optimization problems has been limited. Our study begins by determining the best activation function … Read more

    A Single-Loop Algorithm for Decentralized Bilevel Optimization

  • Youran Dong
  • Shiqian Ma
  • Junfeng Yang
  • Chao Yin
    • Categories Nonlinear Optimization, Optimization in Data Science Tags ,

    Bilevel optimization has gained significant attention in recent years due to its broad applications in machine learning. This paper focuses on bilevel optimization in decentralized networks and proposes a novel single-loop algorithm for solving decentralized bilevel optimization with a strongly convex lower-level problem. Our approach is a fully single-loop method that approximates the hypergradient using … Read more

    Relaxation strength for multilinear optimization: McCormick strikes back

  • Matthias Walter
    • Categories (Mixed) Integer Linear Programming, Nonlinear Optimization, Polyhedra Tags , , , , ,

    We consider linear relaxations for multilinear optimization problems. In a recent paper, Khajavirad proved that the extended flower relaxation is at least as strong as the relaxation of any recursive McCormick linearization (Operations Research Letters 51 (2023) 146-152). In this paper we extend the result to more general linearizations, and present a simpler proof. Moreover, … Read more