Convex and Nonsmooth Optimization

Minimization of nonsmooth nonconvex functions using inexact evaluations and its worst-case complexity

  • Serge Gratton
  • Philippe L. Toint
  • Ehouarn Simon
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization, Unconstrained Optimization Tags , , , ,

    An adaptive regularization algorithm using inexact function and derivatives evaluations is proposed for the solution of composite nonsmooth nonconvex optimization. It is shown that this algorithm needs at most O(|log(epsilon)|.epsilon^{-2}) evaluations of the problem’s functions and their derivatives for finding an $\epsilon$-approximate first-order stationary point. This complexity bound therefore generalizes that provided by [Bellavia, Gurioli, … Read more

    Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron

  • Francis Bach
  • Mark Schmidt
  • Sharan Vaswani
    • Categories Convex Optimization, Generalized Convexity/Monoticity Tags , , ,

    Modern machine learning focuses on highly expressive models that are able to fit or interpolate the data completely, resulting in zero training loss. For such models, we show that the stochastic gradients of common loss functions satisfy a strong growth condition. Under this condition, we prove that constant step-size stochastic gradient descent (SGD) with Nesterov … Read more

    Weak subgradient algorithm for solving nonsmooth nonconvex unconstrained optimization problems

  • Gulcin Dinc Yalcin
  • Refail Kasimbeyli
    • Categories Nonsmooth Optimization, Unconstrained Optimization Tags

    This paper presents a weak subgradient based method for solving nonconvex unconstrained optimization problems. The method uses a weak subgradient of the objective function at a current point, to generate a new one at every iteration. The concept of the weak subgradient is based on the idea of using supporting cones to the graph of … Read more

    An optimal control theory for accelerated optimization

  • Isaac Ross
    • Categories Convex and Nonsmooth Optimization, Unconstrained Optimization Tags , , , , , ,

    Accelerated optimization algorithms can be generated using a double-integrator model for the search dynamics imbedded in an optimal control problem. CitationunpublishedArticleDownload View PDF

    Exploiting Sparsity for Semi-Algebraic Set Volume Computation

  • Didier Henrion
  • Jean-Bernard Lasserre
  • Matteo Tacchi
  • Tillmann Weisser
    • Categories Convex Optimization, Semi-definite Programming

    We provide a systematic deterministic numerical scheme to approximate the volume (i.e. the Lebesgue measure) of a basic semi-algebraic set whose description follows a sparsity pattern. As in previous works (without sparsity), the underlying strategy is to consider an infinite-dimensional linear program on measures whose optimal value is the volume of the set. This is … Read more

    Subdifferentials and SNC property of scalarization functionals with uniform level sets and applications

  • Christiane Tammer
  • Bao Q. Truong
    • Categories Constrained Nonlinear Optimization, Multi-Criteria Optimization, Nonsmooth Optimization

    This paper deals with necessary conditions for minimal solutions of constrained and unconstrained optimization problems with respect to general domination sets by using a well-known nonlinear scalarization functional with uniform level sets (called Gerstewitz’ functional in the literature). The primary objective of this work is to establish revised formulas for basic and singular subdifferentials of … Read more

    Active-set Newton methods and partial smoothness

  • Adrian Lewis
  • Calvin Wylie
    • Categories Complementarity and Variational Inequalities, Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , ,

    Diverse optimization algorithms correctly identify, in finite time, intrinsic constraints that must be active at optimality. Analogous behavior extends beyond optimization to systems involving partly smooth operators, and in particular to variational inequalities over partly smooth sets. As in classical nonlinear programming, such active-set structure underlies the design of accelerated local algorithms of Newton type. … Read more

    Generalized subdifferentials of spectral functions over Euclidean Jordan algebras

  • Bruno F. Lourenço
  • Akiko Takeda
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    This paper is devoted to the study of generalized subdifferentials of spectral functions over Euclidean Jordan algebras. Spectral functions appear often in optimization problems playing the role of “regularizer”, “barrier”, “penalty function” and many others. We provide formulae for the regular, approximate and horizon subdifferentials of spectral functions. In addition, under local lower semicontinuity, we … Read more

    The condition number of a function relative to a set

  • David H. Gutman
  • Javier F. Peña
    • Categories Convex Optimization Tags , , ,

    The condition number of a differentiable convex function, namely the ratio of its smoothness to strong convexity constants, is closely tied to fundamental properties of the function. In particular, the condition number of a quadratic convex function is the square of the aspect ratio of a canonical ellipsoid associated to the function. Furthermore, the condition … Read more

    Fast Robust Methods for Singular State-Space Models

  • Aleksandr Y. Aravkin
  • James V. Burke
  • Sarah Webster
  • Jonathan Jonker
  • Gianluigi Pillonetto
    • Categories Control Applications, Nonsmooth Optimization, Statistics Tags , ,

    State-space models are used in a wide range of time series analysis applications. Kalman filtering and smoothing are work-horse algorithms in these settings. While classic algorithms assume Gaussian errors to simplify estimation, recent advances use a broad range of optimization formulations to allow outlier-robust estimation, as well as constraints to capture prior information. Here we … Read more