Convex and Nonsmooth Optimization

Normalized stochastic proximal approximation methods for nonsmooth composite optimization under heavy-tailed noise

  • Chunhao Han
  • Xiao Wang
  • Pengxiang Xu
  • Jin Zhang
    • Categories Nonlinear Optimization, Nonsmooth Optimization

    In this paper, we study nonsmooth composite optimization problems under heavy-tailed noise, with the objective being a summation of a nested function and a nonsmooth convex regularizer. We propose stochastic proximal approximation methods incorporating a normalization technique to handle the potential challenges caused by the nonsmooth regularizer and heavy-tailed noise. For the case where the … Read more

    Function-free Optimization via Comparison Oracles

  • Katya Scheinberg
  • Zikai Xiong
    • Categories Convex Optimization, Nonlinear Optimization Tags , , ,

    In this work, we study optimization specified only through a comparison oracle: given two points, it reports which one is preferred. We call it function-free optimization because we do not assume access to, nor the existence of, a canonical application-given objective function. Instead, our goal is to find a most-preferred feasible point, which we call … Read more

    Inexact Cubic Regularization Method with Adaptive Reuse of Hessian Approximations

  • Max L. N. Gonçalves
  • vilmargehlen
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    This work introduces an inexact cubic regularization method with adaptive reuse of Hessian approximations to solve general non-convex optimization problems. In the proposed approach, the gradient is computed inexactly and updated at every iteration, whereas the Hessian approximation is updated at a specific iteration and then reused for $m$ subsequent iterations (a lazy strategy), where … Read more

    Inexact proximal point method for piecewise-star-convex function

  • Orizon Pereira Ferreira
  • Max L. N. Gonçalves
  • Jefferson Gonçalves Melo
    • Categories Convex and Nonsmooth Optimization, Nonsmooth Optimization Tags , ,

    We propose and analyze an inexact proximal point method for minimizing locally Lipschitz functions on Euclidean spaces with a piecewise star-convex structure. More precisely, the space is covered by finitely many closed convex sets, and on each set the objective function satisfies a star-convex inequality with respect to the minimizers of its restriction. This class … Read more

    Supervised feature selection via multiobjective programming and its application in the medical field

  • Pham Thi Hoai
    • Categories Applications - OR and Management Sciences, Constrained Nonlinear Optimization, Convex and Nonsmooth Optimization, Convex Optimization, Other Topics Tags , , , ,

    In this study, we model the supervised feature selection problem using a novel approach: convex bi-objective optimization. Traditional methods have addressed this problem by maximizing relevance to class labels and minimizing redundancy among features. Recently, Wang et al. [30] formulated this problem as a single-objective convex optimization, yielding only a unique solution. Unlike that, we … Read more

    Second-Order Optimality Conditions for Bilevel Optimization Problems Using Parabolic Directional Derivatives

  • Bhuvnesh Khatana
    • Categories Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Convex Optimization

    This paper studies the second-order properties of a class of inequality-constrained bilevel programming problems. First-order optimality conditions for the existence of solutions to bilevel optimization problems are derived using the first-order directional derivative of the optimal solution function of the lower-level problem in the seminal paper by Dempe (1992). In this work, we prove that … Read more

    Convergence of the Frank-Wolfe Algorithm for Monotone Variational Inequalities

  • Matthew Hough
    • Categories Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Convex Optimization Tags , , ,

    We consider the Frank-Wolfe algorithm for solving variational inequalities over compact, convex sets under a monotone \(C^1\) operator and vanishing, nonsummable step sizes. We introduce a continuous-time interpolation of the discrete iteration and use tools from dynamical systems theory to analyze its asymptotic behavior. This allows us to derive convergence results for the original discrete … Read more

    Negative Momentum for Convex-Concave Optimization

  • Henry Shugart
  • Shuyi Wang
  • Jason M. Altschuler
    • Categories Convex Optimization

    This paper revisits momentum in the context of min-max optimization. Momentum is a celebrated mechanism for accelerating gradient dynamics in settings like convex minimization, but its direct use in min-max optimization makes gradient dynamics diverge. Surprisingly, Gidel et al. 2019 showed that negative momentum can help fix convergence. However, despite these promising initial results and … Read more

    A Nesterov-Accelerated Primal-Dual Splitting Algorithm for Convex Nonsmooth Optimization

  • Laurent Condat
  • Peter Richtarik
    • Categories Convex and Nonsmooth Optimization

    We investigate the integration of Nesterov-type acceleration into primal-dual methods for structured convex optimization. While proximal splitting algorithms efficiently handle composite problems of the form min_x f(x) + g(x) + h(Kx), accelerating their convergence with respect to the smooth term f is notoriously challenging due to the rotational dynamics in the primal-dual space. In this … Read more

    A unified framework for inexact adaptive stepsizes in the gradient methods, the conjugate gradient methods and the quasi-Newton methods for strictly convex quadratic optimization

  • Zexian Liu
    • Categories Convex Optimization, Quadratic Programming, Unconstrained Optimization Tags , , , ,

    The inexact adaptive stepsizes for the conjugate gradient method and  the quasi-Newton method are very rare. The exact stepsizes in the gradient method, the conjugate gradient method and the  quasi-Newton method for strictly convex quadratic optimization have a unified framework, while the unified framework for inexact adaptive stepsizes  in the gradient method, the conjugate gradient … Read more