Nonlinear Optimization

Subgradient Regularization: A Descent-Oriented Subgradient Method for Nonsmooth Optimization

  • Hanyang Li
  • Ying Cui
    • Categories Convex and Nonsmooth Optimization, Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    In nonsmooth optimization, a negative subgradient is not necessarily a descent direction, making the design of convergent descent methods based on zeroth-order and first-order information a challenging task. The well-studied bundle methods and gradient sampling algorithms construct descent directions by aggregating subgradients at nearby points in seemingly different ways, and are often complicated or lack … Read more

    An inexact alternating projection method with application to matrix completion

  • Stefania Bellavia
  • Simone Rebegoldi
  • Mattia Silei
    • Categories Nonlinear Optimization, Other Topics Tags , , , ,

    We develop and analyze an inexact regularized alternating projection method for nonconvex feasibility problems. Such a method employs inexact projections on one of the two sets, according to a set of well-defined conditions. We prove the global convergence of the algorithm, provided that a certain merit function satisfies the Kurdyka-Lojasiewicz property on its domain. The … Read more

    Steepest descent method using novel adaptive stepsizes for unconstrained nonlinear multiobjective programming

  • Pham Thi Hoai
    • Categories Convex Optimization, Multi-Criteria Optimization, Nonlinear Optimization Tags , , , ,

    We propose new adaptive strategies to compute stepsizes for the steepest descent method to solve unconstrained nonlinear multiobjective optimization problems without employing any linesearch procedure. The resulting algorithms can be applied to a wide class of nonconvex unconstrained multi-criteria optimization problems satisfying a global Lipschitz continuity condition imposed on the gradients of all objectives. In … Read more

    Fast Stochastic Second-Order Adagrad for Nonconvex Bound-Constrained Optimization

  • Stefania Bellavia
  • Serge Gratton
  • Benedetta Morini
  • Philippe L. Toint
    • Categories Bound-constrained Optimization, Data Science Algorithms, Nonlinear Optimization Tags , , , , , ,

    ADAGB2, a generalization of the Adagrad algorithm for stochastic optimization is introduced, which is also applicable to bound-constrained problems and capable of using second-order information when available. It is shown that, given  delta in (0,1) and epsilon in (0,1], the ADAGB2 algorithm needs at most O(epsilon^{-2}) iterations to ensure an epsilon-approximate first-order critical point of … Read more

    A Fast Newton Method Under Local Lipschitz Smoothness

  • Serge Gratton
  • Sadok Jerad
  • Philippe L. Toint
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    A new, fast second-order method is proposed that achieves the optimal \(\mathcal{O}\left(|\log(\epsilon)|\epsilon^{-3/2}\right) \) complexity to obtain first-order $\epsilon$-stationary points. Crucially, this is deduced without assuming the standard global Lipschitz Hessian continuity condition, but onlyusing an appropriate local smoothness requirement. The algorithm exploits Hessian information to compute a Newton step and a negative curvature step when … Read more

    An adaptive single-loop stochastic penalty method for nonconvex constrained stochastic optimization

  • Shiji Zuo
  • Xiao Wang
  • Hao Wang
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization

    Adaptive update schemes for penalty parameters are crucial to enhancing robustness and practical applicability of penalty methods for constrained optimization. However, in the context of general constrained stochastic optimization, additional challenges arise due to the randomness introduced by adaptive penalty parameters. To address these challenges, we propose an Adaptive Single-loop Stochastic Penalty method (AdaSSP) in … Read more

    Sensitivity analysis for parametric nonlinear programming: A tutorial

  • François Pacaud
    • Categories Constrained Nonlinear Optimization, Nonlinear Optimization Tags ,

    This tutorial provides an overview of the current state-of-the-art in the sensitivity analysis for nonlinear programming. Building upon the fundamental work of Fiacco, it derives the sensitivity of primal-dual solutions for regular nonlinear programs and explores the extent to which Fiacco’s framework can be extended to degenerate nonlinear programs with non-unique dual solutions. The survey … Read more

    The improvement function in branch-and-bound methods for complete global optimization

  • Stefan Schwarze
  • Oliver Stein
  • Peter Kirst
    • Categories Constrained Nonlinear Optimization, Global Optimization Tags , , , ,

    We present a new spatial branch-and-bound approach for treating optimization problems with nonconvex inequality constraints. It is able to approximate the set of all global minimal points in case of solvability, and else to detect infeasibility. The new technique covers the nonconvex constraints by means of an improvement function which, although nonsmooth, can be treated … Read more

    IPAS: An Adaptive Sample Size Method for Weighted Finite Sum Problems with Linear Equality Constraints

  • Nataša Krejić
  • Nataša Krklec Jerinkić
  • Sanja Rapajić
  • Luka Rutešić
    • Categories Constrained Nonlinear Optimization, Stochastic Programming Tags , , , , ,

    Optimization problems with the objective function in the form of weighted sum and linear equality constraints are considered. Given that the number of local cost functions can be large as well as the number of constraints, a stochastic optimization method is proposed. The method belongs to the class of variable sample size first order methods, … Read more

    The improvement function reformulation for graphs of minimal point mappings

  • Stefan Schwarze
  • Oliver Stein
    • Categories Constrained Nonlinear Optimization, Game Theory, Global Optimization Theory Tags , , , , ,

    Graphs of minimal point mappings of parametric optimization problems appear in the definition of feasible sets of bilevel optimization problems and of semi-infinite optimization problems, and the intersection of multiple such graphs defines (generalized) Nash equilibria. This paper shows how minimal point graphs of nonconvex parametric optimization problems can be written with the help of … Read more