Nonlinear Optimization

Normalized stochastic proximal approximation methods for nonsmooth composite optimization under heavy-tailed noise

  • Chunhao Han
  • Xiao Wang
  • Pengxiang Xu
  • Jin Zhang
    • Categories Nonlinear Optimization, Nonsmooth Optimization

    In this paper, we study nonsmooth composite optimization problems under heavy-tailed noise, with the objective being a summation of a nested function and a nonsmooth convex regularizer. We propose stochastic proximal approximation methods incorporating a normalization technique to handle the potential challenges caused by the nonsmooth regularizer and heavy-tailed noise. For the case where the … Read more

    Function-free Optimization via Comparison Oracles

  • Katya Scheinberg
  • Zikai Xiong
    • Categories Convex Optimization, Nonlinear Optimization Tags , , ,

    In this work, we study optimization specified only through a comparison oracle: given two points, it reports which one is preferred. We call it function-free optimization because we do not assume access to, nor the existence of, a canonical application-given objective function. Instead, our goal is to find a most-preferred feasible point, which we call … Read more

    Inexact Cubic Regularization Method with Adaptive Reuse of Hessian Approximations

  • Max L. N. Gonçalves
  • vilmargehlen
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    This work introduces an inexact cubic regularization method with adaptive reuse of Hessian approximations to solve general non-convex optimization problems. In the proposed approach, the gradient is computed inexactly and updated at every iteration, whereas the Hessian approximation is updated at a specific iteration and then reused for $m$ subsequent iterations (a lazy strategy), where … Read more

    Accuracy and Relationships of Quadratic Models in Derivative-free Optimization

  • Yiwen Chen
  • Warren Hare
  • Lindon Roberts
    • Categories Nonlinear Optimization Tags , ,

    We study three quadratic models in model-based derivative-free optimization: the minimum norm (MN), minimum Frobenius norm (MFN), and quadratic generalized simplex derivative (QS) models. Despite their widespread use, their approximation accuracy and relationships have not been systematically explored. We establish fully linear error bounds for all three models, removing the uniformly bounded model Hessian assumption … Read more

    Supervised feature selection via multiobjective programming and its application in the medical field

  • Pham Thi Hoai
    • Categories Applications - OR and Management Sciences, Constrained Nonlinear Optimization, Convex and Nonsmooth Optimization, Convex Optimization, Other Topics Tags , , , ,

    In this study, we model the supervised feature selection problem using a novel approach: convex bi-objective optimization. Traditional methods have addressed this problem by maximizing relevance to class labels and minimizing redundancy among features. Recently, Wang et al. [30] formulated this problem as a single-objective convex optimization, yielding only a unique solution. Unlike that, we … Read more

    Out-of-the-Box Global Optimization for Packing Problems: New Models and Improved Solutions

  • Timo Berthold
  • Dominik Kamp
  • Gioni Mexi
  • Sebastian Pokutta
  • Imre Pólik
    • Categories Global Optimization, Nonlinear Optimization

    Recent LLM-driven discoveries have renewed interest in geometric packing problems. In this paper, we study several classes of such packing problems through the lens of modern global nonlinear optimization. Starting from comparatively direct nonlinear formulations, we consider packing circles in squares and fixed-perimeter rectangles, packing circles into minimum-area ellipses, packing regular polygons into regular polygons, … Read more

    Stochastic Three Points Method with an Inexact Oracle and Its Application to Steady-State Optimization

  • Geovani Nunes Grapiglia
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    We consider unconstrained derivative-free optimization problems in which only inexact function evaluations are available. Specifically, we study the setting where the oracle returns function values with partially controllable inexactness, with the error bounded linearly by a user-specified accuracy parameter, but with an unknown proportionality constant. This framework captures optimization problems arising from approximate simulations or … Read more

    A unified framework for inexact adaptive stepsizes in the gradient methods, the conjugate gradient methods and the quasi-Newton methods for strictly convex quadratic optimization

  • Zexian Liu
    • Categories Convex Optimization, Quadratic Programming, Unconstrained Optimization Tags , , , ,

    The inexact adaptive stepsizes for the conjugate gradient method and  the quasi-Newton method are very rare. The exact stepsizes in the gradient method, the conjugate gradient method and the  quasi-Newton method for strictly convex quadratic optimization have a unified framework, while the unified framework for inexact adaptive stepsizes  in the gradient method, the conjugate gradient … Read more

    Accuracy Certificates for Convex Optimization at Accelerated Rates via Primal-Dual Averaging

  • Matthew X. Burns
  • Jiaming Liang
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization Tags , , , ,

    Many works in convex optimization provide rates for achieving a small primal gap. However, this quantity is typically unavailable in practice. In this work, we show that solving a regularized surrogate with algorithms based on simple primal-dual averaging provides non-asymptotic convergence guarantees for a computable optimality certificate. We first analyze primal and dual methods based … Read more

    A semi-smooth Newton method for the nonlinear conic problem with generalized simplicial cones

  • Nicolas F. Armijo
  • Yunier Bello-Cruz
  • Gabriel Haeser
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags , , , , ,

    In this work we develop and analyze a semi-smooth Newton method for the general non- linear conic programming problem. In particular, we study the problem with a generalized simplicial cone, i.e., the image of a symmetric cone under a linear mapping. We generalize Robinson’s normal equations to a conic setting, yielding what we call the … Read more