Nonlinear Optimization

First-order Methods for Unconstrained Vector Optimization Problems: A Unified Majorization-Minimization Perspective

  • Jian Chen
    • Categories Multi-Criteria Optimization, Nonlinear Optimization, Stochastic Programming Tags , , , ,

    In this paper, we develop a unified majorization-minimization scheme and convergence analysis with first-order surrogate functions for unconstrained vector optimization problems (VOPs). By selecting different surrogate functions, the unified method can be reduced to various existing first-order methods. The unified convergence analysis reveals that the slow convergence of the steepest descent method is primarily attributed … Read more

    Global Optimization for Combinatorial Geometry Problems Revisited in the Era of LLMs

  • Timo Berthold
  • Dominik Kamp
  • Gioni Mexi
  • Sebastian Pokutta
  • Imre Pólik
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Optimization Software and Modeling Systems Tags , , ,

    Recent progress in LLM-driven algorithm discovery, exemplified by DeepMind’s AlphaEvolve, has produced new best-known solutions for a range of hard geometric and combinatorial problems. This raises a natural question: to what extent can modern off-the-shelf global optimization solvers match such results when the problems are formulated directly as nonlinear optimization problems (NLPs)? We revisit a … Read more

    A Majorization-Minimization approach for multiclass classification in a big data scenario

  • Giorgia Franchini
  • Federica Porta
  • Émilie Chouzenoux
  • Jean-Christophe Pesquet
  • filippo.camellini
    • Categories Convex Optimization, Data Science Algorithms, Unconstrained Optimization Tags , , , , ,

    This work presents a novel optimization approach for training linear classifiers in multiclass classification tasks, when focusing on a regularized and smooth Weston-Watkins support vector machine (SVM) model. We propose a Majorization-Minimization (MM) algorithm to solve the resulting, Lipschitz-differentiable, optimization problem. To enhance scalability of the algorithm when tackling large datasets, we introduce an incremental … Read more

    The Maximum Clique Problem under Adversarial Uncertainty: a min-max approach

  • Giovanni Spisso
  • Immanuel M. Bomze
  • Francesco Rinaldi
  • Chiara Faccio
    • Categories Nonlinear Optimization, Nonsmooth Optimization, Other Topics Tags , , ,

    We analyze the problem of identifying large cliques in graphs that are affected by adversarial uncertainty. More specifically, we consider a new formulation, namely the adversarial maximum clique problem, which extends the classical maximum-clique problem to graphs with edges strategically perturbed by an adversary. The proposed mathematical model is thus formulated as a two-player zero-sum … Read more

    A speed up strategy for gradient methods

  • Anna De Magistris
  • Serena Crisci
  • Valentina De Simone
  • Gerardo Toraldo
    • Categories Convex Optimization, Quadratic Programming Tags , , ,

    In this paper, we propose a new acceleration strategy for gradient-based methods applied to strictly convex Quadratic Programming (QP) problems. The strategy consists in performing, at selected iterations, minimization steps along alternative descent directions or even within low-dimensional affine subspaces. In particular, considering the contribution of the linear and quadratic part of the objective function … Read more

    An Inexact Modified Quasi-Newton Method for Nonsmooth Regularized Optimization

  • Nathan Allaire
  • Sébastien Le Digabel
  • Dominique Orban
    • Categories Data Science Algorithms, Nonlinear Optimization, Nonsmooth Optimization Tags , , , , , , , ,

    We introduce method iR2N, a modified proximal quasi-Newton method for minimizing the sum of a \(C^1\) function \(f\) and a lower semi-continuous prox-bounded \(h\) that permits inexact evaluations of \(f\), \(\nabla f\) and of the relevant proximal operators. Both \(f\) and \(h\) may be nonconvex. In applications where the proximal operator of \(h\) is not … Read more

    The Convexity Zoo: A Taxonomy of Function Classes in Optimization

  • Abbas Khademi
    • Categories Constrained Nonlinear Optimization, Generalized Convexity/Monoticity, Nonlinear Optimization Tags , , , ,

    The tractability of optimization problems depends critically on structural properties of the objective function. Convexity guarantees global optimality of local solutions and enables polynomial-time algorithms under mild assumptions, but many problems arising in modern applications—particularly in machine learning—are inherently nonconvex. Remarkably, a large class of such problems remains amenable to efficient optimization due to additional … Read more

    A Proximal-Gradient Method for Solving Regularized Optimization Problems with General Constraints

  • Daniel P. Robinson
  • Frank E. Curtis
  • Xiaoyi Qu
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , , , , ,

    We propose, analyze, and test a proximal-gradient method for solving regularized optimization problems with general constraints. The method employs a decomposition strategy to compute trial steps and uses a merit function to determine step acceptance or rejection. Under various assumptions, we establish a worst-case iteration complexity result, prove that limit points are first-order KKT points, … Read more

    Fast and Simple Multiclass Data Segmentation: An Eigendecomposition and Projection-Free Approach

  • Chiara Faccio
  • Margherita Porcelli
  • Francesco Rinaldi
  • Martin Stoll
    • Categories Data Science Algorithms, Data Science Applications, Nonlinear Optimization

    Graph-based machine learning has seen an increased interest over the last decade with many connections to other fields of applied mathematics. Learning based on partial differential equations, such as the phase-field Allen-Cahn equation, allows efficient handling of semi-supervised learning approaches on graphs. The numerical solution of the graph Allen-Cahn equation via a convexity splitting or … Read more

    On constraint qualifications for lower-level sets and an augmented Lagrangian method

  • Roberto Andreani
  • Gabriel Haeser
  • Mariana da Rosa
  • Daiana O. Santos
    • Categories Nonlinear Optimization Tags , , ,

    In this paper we consider an augmented Lagrangian method with general lower-level constraints, that is, where some of the constraints are penalized while others are kept as subproblem constraints. Motivated by some recent results on optimization problems on manifolds, we present a general theory of global convergence when a feasible approximate KKT point is found … Read more