Nonlinear Optimization – Page 6 – Optimization Online

A Majorization-Minimization approach for multiclass classification in a big data scenario

Published: 2026/01/08, Updated: 2026/01/09

Convex Optimization, Data Science Algorithms, Unconstrained Optimization Foundation Models, Incremental Minimization, machine learning, majorization-minimization method, Multiclass Classification, Support Vector Machine

This work presents a novel optimization approach for training linear classifiers in multiclass classification tasks, when focusing on a regularized and smooth Weston-Watkins support vector machine (SVM) model. We propose a Majorization-Minimization (MM) algorithm to solve the resulting, Lipschitz-differentiable, optimization problem. To enhance scalability of the algorithm when tackling large datasets, we introduce an incremental … Read more

The Maximum Clique Problem under Adversarial Uncertainty: a min-max approach

Published: 2026/01/08

Nonlinear Optimization, Nonsmooth Optimization, Other Topics adversarial uncertainty, maximum clique, nonsmooth optimization, projection-free methods

We analyze the problem of identifying large cliques in graphs that are affected by adversarial uncertainty. More specifically, we consider a new formulation, namely the adversarial maximum clique problem, which extends the classical maximum-clique problem to graphs with edges strategically perturbed by an adversary. The proposed mathematical model is thus formulated as a two-player zero-sum … Read more

A speed up strategy for gradient methods

Published: 2026/01/07

Convex Optimization, Quadratic Programming acceleration strategies, bound constrained quadratic programming, convex quadratic programming, gradient methods

In this paper, we propose a new acceleration strategy for gradient-based methods applied to strictly convex Quadratic Programming (QP) problems. The strategy consists in performing, at selected iterations, minimization steps along alternative descent directions or even within low-dimensional affine subspaces. In particular, considering the contribution of the linear and quadratic part of the objective function … Read more

An Inexact Modified Quasi-Newton Method for Nonsmooth Regularized Optimization

Published: 2026/01/05

Nathan Allaire

Sébastien Le Digabel

Dominique Orban

Data Science Algorithms, Nonlinear Optimization, Nonsmooth Optimization composite optimization, inexact evaluations, inexact proximal operator, modified quasi-Newton method, nonconvex optimization, nonsmooth optimization, proximal gradient method, proximal quasi-newton method, regularized optimization

We introduce method iR2N, a modified proximal quasi-Newton method for minimizing the sum of a \(C^1\) function \(f\) and a lower semi-continuous prox-bounded \(h\) that permits inexact evaluations of \(f\), \(\nabla f\) and of the relevant proximal operators. Both \(f\) and \(h\) may be nonconvex. In applications where the proximal operator of \(h\) is not … Read more

The Convexity Zoo: A Taxonomy of Function Classes in Optimization

Published: 2026/01/01, Updated: 2026/04/18

Abbas Khademi

Constrained Nonlinear Optimization, Generalized Convexity/Monoticity, Nonlinear Optimization generalized convexity, nonconvex optimization, quasar-convexity, star-convexity, structured nonconvexity

The tractability of optimization problems depends critically on structural properties of the objective function. Convexity guarantees global optimality of local solutions and enables polynomial-time algorithms under mild assumptions, but many problems arising in modern applications—particularly in machine learning—are inherently nonconvex. Remarkably, a large class of such problems remains amenable to efficient optimization due to additional … Read more

A Proximal-Gradient Method for Solving Regularized Optimization Problems with General Constraints

Published: 2025/12/27

Daniel P. Robinson

Frank E. Curtis

Xiaoyi Qu

Constrained Nonlinear Optimization, Nonlinear Optimization, Nonsmooth Optimization composite optimization, constrained optimization, nonconvex optimization, nonlinear optimization, proximal gradient method, regularization, worst-case iteration-complexity

We propose, analyze, and test a proximal-gradient method for solving regularized optimization problems with general constraints. The method employs a decomposition strategy to compute trial steps and uses a merit function to determine step acceptance or rejection. Under various assumptions, we establish a worst-case iteration complexity result, prove that limit points are first-order KKT points, … Read more

Fast and Simple Multiclass Data Segmentation: An Eigendecomposition and Projection-Free Approach

Published: 2025/12/18

Data Science Algorithms, Data Science Applications, Nonlinear Optimization

Graph-based machine learning has seen an increased interest over the last decade with many connections to other fields of applied mathematics. Learning based on partial differential equations, such as the phase-field Allen-Cahn equation, allows efficient handling of semi-supervised learning approaches on graphs. The numerical solution of the graph Allen-Cahn equation via a convexity splitting or … Read more

On constraint qualifications for lower-level sets and an augmented Lagrangian method

Published: 2025/12/18, Updated: 2026/01/02

Nonlinear Optimization constraint qualifications, global convergence, nonlinear optimization, ugmented Lagrangian methods

In this paper we consider an augmented Lagrangian method with general lower-level constraints, that is, where some of the constraints are penalized while others are kept as subproblem constraints. Motivated by some recent results on optimization problems on manifolds, we present a general theory of global convergence when a feasible approximate KKT point is found … Read more

Robust optimality for nonsmooth mathematical programs with equilibrium constraints under data uncertainty

Published: 2025/12/11

Vivek Laha

Applications - OR and Management Sciences, Constrained Nonlinear Optimization, Convex and Nonsmooth Optimization, Convex Optimization, Nonlinear Optimization, Nonsmooth Optimization, Robust Optimization robust optimization

We develop a unified framework for robust nonsmooth optimization problems with equilibrium constraints (UNMPEC). As a foundation, we study a robust nonsmooth nonlinear program with uncertainty in both the objective function and the inequality constraints (UNP). Using Clarke subdifferentials, we establish Karush–Kuhn–Tucker (KKT)–type necessary optimality conditions under an extended no–nonzero–abnormal–multiplier constraint qualification (ENNAMCQ). When the … Read more

New Results on the Polyak Stepsize: Tight Convergence Analysis and Universal Function Classes

Published: 2025/12/11

Convex and Nonsmooth Optimization, Convex Optimization, Nonlinear Optimization, Nonsmooth Optimization adaptive first-order methods, Lower bound

In this paper, we revisit a classical adaptive stepsize strategy for gradient descent: the Polyak stepsize (PolyakGD), originally proposed in Polyak (1969). We study the convergence behavior of PolyakGD from two perspectives: tight worst-case analysis and universality across function classes. As our first main result, we establish the tightness of the known convergence rates of … Read more