Nonsmooth Optimization

A Local MM Subspace Method for Solving Constrained Variational Problems in Image Recovery

  • Émilie Chouzenoux
  • Ségolène Martin
  • Jean-Christophe Pesquet
    • Categories Constrained Nonlinear Optimization, Convex Optimization, Nonsmooth Optimization Tags , , , , , ,

    This article introduces a new Penalized Majorization-Minimization Subspace algorithm (P-MMS) for solving smooth, constrained optimization problems. In short, our approach consists of embedding a subspace algorithm in an inexact exterior penalty procedure. The subspace strategy, combined with a Majoration-Minimization step-size search, takes great advantage of the smoothness of the penalized cost function, while the penalty … Read more

    Subgradient methods near active manifolds: saddle point avoidance, local convergence, and asymptotic normality

  • Damek Davis
  • Dmitriy Drusvyatskiy
  • Liwei Jiang
    • Categories Nonsmooth Optimization

    Nonsmooth optimization problems arising in practice, whether in signal processing, statistical estimation, or modern machine learning, tend to exhibit beneficial smooth substructure: their domains stratify into “active manifolds” of smooth variation, which common proximal algorithms “identify” in finite time. Identification then entails a transition to smooth dynamics, and permits the use of second-order information for … Read more

    On Properties of Univariate Max Functions at Local Maximizers

  • Tim Mitchell
  • Michael L. Overton
    • Categories Nonsmooth Optimization Tags , , , ,

    More than three decades ago, Boyd and Balakrishnan established a regularity result for the two-norm of a transfer function at maximizers. Their result extends easily to the statement that the maximum eigenvalue of a univariate real analytic Hermitian matrix family is twice continuously differentiable, with Lipschitz second derivative, at all local maximizers, a property that … Read more

    An abstract convergence framework with application to inertial inexact forward-backward methods

  • Silvia Bonettini
  • Peter Ochs
  • Marco Prato
  • Simone Rebegoldi
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    In this paper we introduce a novel abstract descent scheme suited for the minimization of proper and lower semicontinuous functions. The proposed abstract scheme generalizes a set of properties that are crucial for the convergence of several first-order methods designed for nonsmooth nonconvex optimization problems. Such properties guarantee the convergence of the full sequence of … Read more

    A Semismooth Newton-Type Method for the Nearest Doubly Stochastic Matrix Problem

  • Hao Hu
  • Xinxin Li
  • Henry Wolkowicz
  • Haesol Im
    • Categories Convex and Nonsmooth Optimization, Nonsmooth Optimization Tags , , , ,

    We study a semismooth Newton-type method for the nearest doubly stochastic matrix problem where both differentiability and nonsingularity of the Jacobian can fail. The optimality conditions for this problem are formulated as a system of strongly semismooth functions. We show that the so-called local error bound condition does not hold for this system. Thus the … Read more

    FISTA and Extensions – Review and New Insights

  • Weiwei Kong
  • Jefferson Gonçalves Melo
  • Renato D.C. Monteiro
    • Categories Convex Optimization, Nonsmooth Optimization Tags , ,

    The purpose of this technical report is to review the main properties of an accelerated composite gradient (ACG) method commonly referred to as the Fast Iterative Shrinkage-Thresholding Algorithm (FISTA). In addition, we state a version of FISTA for solving both convex and strongly convex composite minimization problems and derive its iteration complexities to generate iterates … Read more

    Alternative Regularizations for OA Algorithms for Convex MINLP

  • David E. Bernal Neira
  • Ignacio E. Grossmann
  • Jan Kronqvist
  • Zedong Peng
    • Categories (Mixed) Integer Nonlinear Programming, Integer Programming, Nonsmooth Optimization

    In this work, we extend the regularization framework from Kronqvist et al. (https://doi.org/10.1007/s10107-018-1356-3) by incorporating several new regularization functions and develop a regularized single-tree search method for solving convex mixed-integer nonlinear programming (MINLP) problems. We propose a set of regularization functions based on distance-metrics and Lagrangean approximations, used in the projection problem for finding new … Read more

    On the Convergence Results of a class of Nonmonotone Accelerated Proximal Gradient Methods for Nonsmooth and Nonconvex Minimization Problems

  • Hongwei Liu
  • Wang Ting
    • Categories Nonsmooth Optimization Tags , ,

    In this paper, we consider a class of nonsmooth problem that is the sum of a Lipschitz differentiable function and a nonsmooth and proper lower semicontinuous function. We discuss here the convergence rate of the function values for a nonmonotone accelerated proximal gradient method, which proposed in “Huan Li and Zhouchen Lin: Accelerated proximal gradient … Read more

    Local Minimizers of the Crouzeix Ratio: A Nonsmooth Optimization Case Study

  • Michael L. Overton
    • Categories Global Optimization, Nonsmooth Optimization Tags , , ,

    Given a square matrix $A$ and a polynomial $p$, the Crouzeix ratio is the norm of the polynomial on the field of values of $A$ divided by the 2-norm of the matrix $p(A)$. Crouzeix’s conjecture states that the globally minimal value of the Crouzeix ratio is 0.5, regardless of the matrix order and polynomial degree, … Read more

    Radial Duality Part II: Applications and Algorithms

  • Benjamin Grimmer
    • Categories Global Optimization Theory, Nonlinear Optimization, Nonsmooth Optimization

    The first part of this work established the foundations of a radial duality between nonnegative optimization problems, inspired by the work of (Renegar, 2016). Here we utilize our radial duality theory to design and analyze projection-free optimization algorithms that operate by solving a radially dual problem. In particular, we consider radial subgradient, smoothing, and accelerated … Read more