Nonsmooth Optimization

Sharpness and well-conditioning of nonsmooth convex formulations in statistical signal recovery

  • Lijun Ding
  • Alex L. Wang
    • Categories Nonsmooth Optimization, Optimization in Data Science, Statistics Tags , , , , ,

    We study a sample complexity vs. conditioning tradeoff in modern signal recovery problems where convex optimization problems are built from sampled observations. We begin by introducing a set of condition numbers related to sharpness in \(\ell_p\) or Schatten-p norms (\(p\in[1,2]\)) based on nonsmooth reformulations of a class of convex optimization problems, including sparse recovery, low-rank … Read more

    Variational Theory and Algorithms for a Class of Asymptotically Approachable Nonconvex Problems

  • Hanyang Li
  • Ying Cui
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    We investigate a class of composite nonconvex functions, where the outer function is the sum of univariate extended-real-valued convex functions and the inner function is the limit of difference-of-convex functions. A notable feature of this class is that the inner function may fail to be locally Lipschitz continuous. It covers a range of important yet … Read more

    A new single-layer inverse-free fixed-time dynamical system for absolute value equations

  • Xin Han
    • Categories Complementarity and Variational Inequalities, Nonsmooth Optimization Tags , , ,

    In this technical note, a novel single-layer inverse-free fixed-time dynamical system (SIFDS) is proposed to address absolute value equations. The proposed SIFDS directly employs coefficient matrix and absolute value equation function that aims at circumventing matrix inverse operation and achieving fixed-time convergence. The equilibria of the proposed SIFDS is proved to be the unique solution … Read more

    Behavior of Newton-type methods near critical solutions of nonlinear equations with semismooth derivatives

  • Andreas Fischer
  • Alexey F. Izmailov
  • Mario Jelitte
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , ,

    Having in mind singular solutions of smooth reformulations of complementarity problems, arising unavoidably when the solution in question violates strict complementarity, we study the behavior of Newton-type methods near singular solutions of nonlinear equations, assuming that the operator of the equation possesses a strongly semismooth derivative, but is not necessarily twice differentiable. These smoothness restrictions … Read more

    Adaptive Importance Sampling Based Surrogation Methods for Bayesian Hierarchical Models, via Logarithmic Integral Optimization

  • Ziyu He
  • Junyi Liu
  • Jong-Shi Pang
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization, Stochastic Programming Tags , ,

    We explore Maximum a Posteriori inference of Bayesian Hierarchical Models (BHMs) with intractable normalizers, which are increasingly prevalent in contemporary applications and pose computational challenges when combined with nonconvexity and nondifferentiability. To address these, we propose the Adaptive Importance Sampling-based Surrogation method, which efficiently handles nonconvexity and nondifferentiability while improving the sampling approximation of the … Read more

    The complexity of first-order optimization methods from a metric perspective

  • Adrian Lewis
  • Tonghua Tian
    • Categories Nonsmooth Optimization, Other Topics Tags , , , , ,

    A central tool for understanding first-order optimization algorithms is the Kurdyka-Lojasiewicz inequality. Standard approaches to such methods rely crucially on this inequality to leverage sufficient decrease conditions involving gradients or subgradients. However, the KL property fundamentally concerns not subgradients but rather “slope”, a purely metric notion. By highlighting this view, and avoiding any use of … Read more

    First-Order Methods for Nonsmooth Nonconvex Functional Constrained Optimization with or without Slater Points

  • Zhichao Jia
  • Benjamin Grimmer
    • Categories Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    Constrained optimization problems where both the objective and constraints may be nonsmooth and nonconvex arise across many learning and data science settings. In this paper, we show a simple first-order method finds a feasible, ϵ-stationary point at a convergence rate of O(ϵ−4) without relying on compactness or Constraint Qualification (CQ). When CQ holds, this convergence is measured by … Read more

    A New Inexact Proximal Linear Algorithm with Adaptive Stopping Criteria for Robust Phase Retrieval

  • Zhong Zheng
  • Shiqian Ma
  • Lingzhou Xue
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    This paper considers the robust phase retrieval problem, which can be cast as a nonsmooth and nonconvex optimization problem. We propose a new inexact proximal linear algorithm with the subproblem being solved inexactly. Our contributions are two adaptive stopping criteria for the subproblem. The convergence behavior of the proposed methods is analyzed. Through experiments on … Read more

    A descent method for nonsmooth multiobjective optimization problems on Riemannian manifolds

  • Chunming Tang
  • Hao He
  • Jinbao Jian
  • Miantao Chao
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags

    In this paper, a descent method for nonsmooth multiobjective optimization problems on complete Riemannian manifolds is proposed. The objective functions are only assumed to be locally Lipschitz continuous instead of convexity used in existing methods. A necessary condition for Pareto optimality in Euclidean space is generalized to the Riemannian setting. At every iteration, an acceptable … Read more

    On the B-differential of the componentwise minimum of two affine vector functions

  • Jean-Pierre Dussault
  • Jean Charles Gilbert
  • Baptiste Plaquevent-Jourdain
    • Categories Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Nonsmooth Optimization Tags , , , , , , , , , , , , , , ,

    This paper focuses on the description and computation of the B-differential of the componentwise minimum of two affine vector functions. This issue arises in the reformulation of the linear complementarity problem with the Min C-function. The question has many equivalent formulations and we identify some of them in linear algebra, convex analysis and discrete geometry. … Read more