New Nonlinear Conjugate Gradient Methods with Guaranteed Descent for Multi-Objective Optimization

  • Manuel Berkemeier
  • sonntagk
    • Categories Multi-Criteria Optimization, Nonlinear Optimization, Unconstrained Optimization Tags , , , ,

    In this article, we present several examples of special nonlinear conjugate gradient directions for nonlinear (non-convex) multi-objective optimization. These directions provide a descent direction for the objectives, independent of the line-search. This way, we can provide an algorithm with simple, Armijo-like backtracking and prove convergence to first-order critical points. In contrast to other popular conjugate … Read more

    Some Unified Theory for Variance Reduced Prox-Linear Methods

  • Yue Wu
  • Benjamin Grimmer
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , ,

    This work considers the nonconvex, nonsmooth problem of minimizing a composite objective of the form $f(g(x))+h(x)$ where the inner mapping $g$ is a smooth finite summation or expectation amenable to variance reduction. In such settings, prox-linear methods can enjoy variance-reduced speed-ups despite the existence of nonsmoothness. We provide a unified convergence theory applicable to a … Read more

    Computing Weak Counterfactual Explanations for Linear Optimization: A New Class of Bilevel Models and a Tailored Penalty Alternating Direction Method

  • Henri Lefebvre
  • Martin Schmidt
    • Categories Energy, Nonlinear Optimization, Optimization in Data Science Tags , , , ,

    Explainable artificial intelligence is one of the most important trends in modern machine-learning research. The idea is to explain the outcome of a model by presenting a certain change in the input of the model so that the outcome changes significantly. In this paper, we study this question for linear optimization problems as an automated … Read more

    Stochastic first-order methods with multi-extrapolated momentum for highly smooth unconstrained optimization

  • Chuan He
    • Categories Optimization in Data Science, Unconstrained Optimization Tags , , , , ,

    In this paper we consider an unconstrained stochastic optimization problem where the objective function exhibits a high order of smoothness. In particular, we propose a stochastic first-order method (SFOM) with multi-extrapolated momentum, in which multiple extrapolations are performed in each iteration, followed by a momentum step based on these extrapolations. We show that our proposed … Read more

    Randomized Subspace Derivative-Free Optimization with Quadratic Models and Second-Order Convergence

  • Lindon Roberts
  • Coralia Cartis
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , ,

    We consider model-based derivative-free optimization (DFO) for large-scale problems, based on iterative minimization in random subspaces. We provide the first worst-case complexity bound for such methods for convergence to approximate second-order critical points, and show that these bounds have significantly improved dimension dependence compared to standard full-space methods, provided low accuracy solutions are desired and/or … Read more

    A subgradient splitting algorithm for optimization on nonpositively curved metric spaces

  • Ariel Goodwin
  • Adrian Lewis
    • Categories Convex Optimization, Generalized Convexity/Monoticity, Nonsmooth Optimization

    Many of the primal ingredients of convex optimization extend naturally from Euclidean to Hadamard spaces — nonpositively curved metric spaces like Euclidean, Hilbert, and hyperbolic spaces, metric trees, and more general CAT(0) cubical complexes. Linear structure, however, and the duality theory it supports are absent. Nonetheless, we introduce a new type of subgradient for convex … Read more

    Inexact FISTA-like Methods with Adaptive Backtracking

  • Gabriel Grillo
  • Sandra A. Santos
  • Elias Salomão Helou
    • Categories Biomedical Applications, Convex and Nonsmooth Optimization, Convex Optimization Tags , ,

    Accelerated proximal gradient methods have become a useful tool in large-scale convex optimization, specially for variational regularization with non-smooth priors. Prevailing convergence analysis considers that users can perform the proximal and the gradient steps exactly. Still, in some practical applications, the proximal or the gradient steps must be computed inexactly, which can harm convergence speed … Read more

    Addressing Estimation Errors through Robust Portfolio Optimization

  • Gérard Cornuéjols
  • Ozgun Elci
  • Vrishabh Patil
    • Categories Convex Optimization, Quadratic Programming, Robust Optimization Tags , ,

    It is well known that the performance of the classical Markowitz model for portfolio optimization is extremely sensitive to estimation errors on the expected asset returns. Robust optimization mitigates this issue. We focus on ellipsoidal uncertainty sets around a point estimate of the expected asset returns. An important issue is the choice of the parameters … Read more

    A multilevel stochastic regularized first-order method with application to finite sum minimization

  • Margherita Porcelli
  • Elisa Riccietti
  • Filippo Marini
    • Categories Nonlinear Optimization, Stochastic Programming

    In this paper, we propose a multilevel stochastic framework for the solution of nonconvex unconstrained optimization problems. The proposed approach uses random regularized first-order models that exploit an available hierarchical description of the problem, being either in the classical variable space or in the function space, meaning that different levels of accuracy for the objective … Read more

    An inertial Riemannian gradient ADMM for nonsmooth manifold optimization

  • Xiaoquan Wang
    • Categories Convex Optimization, Nonsmooth Optimization, Optimization of Systems modeled by PDEs

    The Alternating Direction Method of Multipliers (ADMM) is widely recognized for its efficiency in solving separable optimization problems. However, its application to optimization on Riemannian manifolds remains a significant challenge. In this paper, we propose a novel inertial Riemannian gradient ADMM (iRG-ADMM) to solve Riemannian optimization problems with nonlinear constraints. Our key contributions are as … Read more