Convex and Nonsmooth Optimization

Interior-point algorithms with full Newton steps for nonsymmetric convex conic optimization

  • Dávid Papp
  • Anita Varga
    • Categories Cone Programming, Convex Optimization Tags , , ,

    We design and analyze primal-dual, feasible interior-point algorithms (IPAs) employing full Newton steps to solve convex optimization problems in standard conic form. Unlike most nonsymmetric cone programming methods, the algorithms presented in this paper require only a logarithmically homogeneous self-concordant barrier (LHSCB) of the primal cone, but compute feasible and \(\varepsilon\)-optimal solutions to both the … Read more

    Provable and Practical Online Learning Rate Adaptation with Hypergradient Descent

  • Wenzhi Gao
    • Categories Convex Optimization, Nonlinear Optimization Tags

    This paper investigates the convergence properties of the hypergradient descent method (HDM), a 25-year-old heuristic originally proposed for adaptive stepsize selection in stochastic first-order methods. We provide the first rigorous convergence analysis of HDM using the online learning framework of [Gao24] and apply this analysis to develop new state-of-the-art adaptive gradient methods with empirical and … Read more

    Mean and variance estimation complexity in arbitrary distributions via Wasserstein minimization

  • Valentio Iverson
  • Stephen A. Vavasis
    • Categories Convex Optimization, Statistics Tags , , , ,

    Parameter estimation is a fundamental challenge in machine learning, crucial for tasks such as neural network weight fitting and Bayesian inference. This paper focuses on the complexity of estimating translation μ∈R^l and shrinkage σ∈R++ parameters for a distribution of the form (1/sigma^l) f_0((x−μ)/σ), where f_0 is a known density in R^l given n samples. We … Read more

    Convergence of Descent Optimization Algorithms under Polyak-Lojasiewicz-Kurdyka Conditions

  • Glaydston Bento
  • Boris S. Mordukhovich
  • Yurii Nesterov
  • Tiago Sousa Mota
    • Categories Nonsmooth Optimization, Unconstrained Optimization

    This paper develops a comprehensive convergence analysis for generic classes of descent algorithms in nonsmooth and nonconvex optimization under several conditions of the Polyak-Lojasiewicz-Kurdyka (PLK) type. Along other results, we prove the finite termination of generic algorithms under the PLK conditions with lower exponents. Specifications are given to establish new convergence rates for inexact reduced … Read more

    A necessary condition for the guarantee of the superiorization method

  • Kay Barshad
  • Yair Censor
  • Walaa M. Moursi
  • Tyler Weames
  • Henry Wolkowicz
    • Categories Biomedical Applications, Constrained Nonlinear Optimization, Convex Optimization Tags , , , , ,

    We study a method that involves principally convex feasibility-seeking and makes secondary efforts of objective function value reduction. This is the well-known superiorization method (SM), where the iterates of an asymptotically convergent iterative feasibility-seeking algorithm are perturbed by objective function nonascent steps. We investigate the question under what conditions a sequence generated by an SM … Read more

    Facial structure of copositive and completely positive cones over a second-order cone

  • Mitsuhiro Nishijima
  • Bruno F. Lourenço
    • Categories Convex and Nonsmooth Optimization, Linear, Cone and Semidefinite Programming Tags , , , , , ,

    We classify the faces of copositive and completely positive cones over a second-order cone and investigate their dimension and exposedness properties. Then we compute two parameters related to chains of faces of both cones. At the end, we discuss some possible extensions of the results with a view toward analyzing the facial structure of general … Read more

    Descent Scheme for a Class of Bilevel Programming Problems

  • Bhuvnesh Khatana
    • Categories Approximation Algorithms, Combinatorial Optimization, Complementarity and Variational Inequalities, Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , ,

    In this paper, a class of bilevel programming problems is studied, in which the lower level is a quadratic programming problem, and the upper level problem consists of a nonlinear objective function with coupling constraints. An iterative process is developed to generate a sequence of points, which converges to the solution of this problem. In … Read more

    Stable Set Polytopes with Rank |V(G)|/3 for the Lovász-Schrijver SDP Operator

  • Yu Hin (Gary) Au
    • Categories Combinatorial Optimization, Convex Optimization, Semi-definite Programming Tags , , , ,

    We study the lift-and-project rank of the stable set polytope of graphs with respect to the Lovász–Schrijver SDP operator \( \text{LS}_+ \) applied to the fractional stable set polytope. In particular, we show that for every positive integer \( \ell \), the smallest possible graph with \( \text{LS}_+ \)-rank \( \ell \) contains \( 3\ell … Read more

    Projected proximal gradient trust-region algorithm for nonsmooth optimization

  • Lindon Roberts
  • Minh N. Dao
  • Hung M. Phan
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    We consider trust-region methods for solving optimization problems where the objective is the sum of a smooth, nonconvex function and a nonsmooth, convex regularizer. We extend the global convergence theory of such methods to include worst-case complexity bounds in the case of unbounded model Hessian growth, and introduce a new, simple nonsmooth trust-region subproblem solver … Read more

    Unifying restart accelerated gradient and proximal bundle methods

  • Jiaming Liang
    • Categories Convex and Nonsmooth Optimization, Convex Optimization Tags , , , ,

    This paper presents a novel restarted version of Nesterov’s accelerated gradient method and establishes its optimal iteration-complexity for solving convex smooth composite optimization problems. The proposed restart accelerated gradient method is shown to be a specific instance of the accelerated inexact proximal point framework introduced in “An accelerated hybrid proximal extragradient method for convex optimization … Read more