Nonlinear Optimization

Inexact restoration with subsampled trust-region methods for finite-sum minimization

  • Stefania Bellavia
  • Nataša Krejić
  • Benedetta Morini
    • Categories Unconstrained Optimization Tags , , , ,

    Convex and nonconvex finite-sum minimization arises in many scientific computing and machine learning applications. Recently, first-order and second-order methods where objective functions, gradients and Hessians are approximated by randomly sampling components of the sum have received great attention. We propose a new trust-region method which employs suitable approximations of the objective function, gradient and Hessian … Read more

    Subdifferentials and SNC property of scalarization functionals with uniform level sets and applications

  • Christiane Tammer
  • Bao Q. Truong
    • Categories Constrained Nonlinear Optimization, Multi-Criteria Optimization, Nonsmooth Optimization

    This paper deals with necessary conditions for minimal solutions of constrained and unconstrained optimization problems with respect to general domination sets by using a well-known nonlinear scalarization functional with uniform level sets (called Gerstewitz’ functional in the literature). The primary objective of this work is to establish revised formulas for basic and singular subdifferentials of … Read more

    Non-asymptotic Results for Langevin Monte Carlo: Coordinate-wise and Black-box Sampling

  • Krishnakumar Balasubramanian
  • Saeed Ghadimi
  • Lingqing Shen
    • Categories Nonlinear Optimization, Statistics Tags , , ,

    Euler-Maruyama and Ozaki discretization of a continuous time diffusion process is a popular technique for sampling, that uses (upto) gradient and Hessian information of the density respectively. The Euler-Maruyama discretization has been used particularly for sampling under the name of Langevin Monte Carlo (LMC) for sampling from strongly log-concave densities. In this work, we make … Read more

    Active-set Newton methods and partial smoothness

  • Adrian Lewis
  • Calvin Wylie
    • Categories Complementarity and Variational Inequalities, Constrained Nonlinear Optimization, Nonsmooth Optimization Tags , ,

    Diverse optimization algorithms correctly identify, in finite time, intrinsic constraints that must be active at optimality. Analogous behavior extends beyond optimization to systems involving partly smooth operators, and in particular to variational inequalities over partly smooth sets. As in classical nonlinear programming, such active-set structure underlies the design of accelerated local algorithms of Newton type. … Read more

    Quasi-Newton Methods for Deep Learning: Forget the Past, Just Sample

  • Albert S. Berahas
  • Majid Jahani
  • Martin Takac
    • Categories Nonlinear Optimization, Unconstrained Optimization Tags , , ,

    We present two sampled quasi-Newton methods: sampled LBFGS and sampled LSR1. Contrary to the classical variants of these methods that sequentially build (inverse) Hessian approximations as the optimization progresses, our proposed methods sample points randomly around the current iterate to produce these approximations. As a result, the approximations constructed make use of more reliable (recent … Read more

    When a maximal angle among cones is nonobtuse

  • Michael Orlitzky
    • Categories Constrained Nonlinear Optimization, Optimization Software and Modeling Systems Tags , , , ,

    Principal angles between linear subspaces have been studied for their application to statistics, numerical linear algebra, and other areas. In 2005, Iusem and Seeger defined critical angles within a single convex cone as an extension of antipodality in a compact set. Then, in 2016, Seeger and Sossa extended that notion to two cones. This was … Read more

    Analysis of the BFGS Method with Errors

  • Richard H. Byrd
  • Jorge Nocedal
  • Yuchen Xie
    • Categories Unconstrained Optimization Tags ,

    The classical convergence analysis of quasi-Newton methods assumes that the function and gradients employed at each iteration are exact. In this paper, we consider the case when there are (bounded) errors in both computations and establish conditions under which a slight modification of the BFGS algorithm with an Armijo-Wolfe line search converges to a neighborhood … Read more

    Generalized subdifferentials of spectral functions over Euclidean Jordan algebras

  • Bruno F. Lourenço
  • Akiko Takeda
    • Categories Nonlinear Optimization, Nonsmooth Optimization Tags , , , ,

    This paper is devoted to the study of generalized subdifferentials of spectral functions over Euclidean Jordan algebras. Spectral functions appear often in optimization problems playing the role of “regularizer”, “barrier”, “penalty function” and many others. We provide formulae for the regular, approximate and horizon subdifferentials of spectral functions. In addition, under local lower semicontinuity, we … Read more

    An Augmented Lagrangian Method for Quasi-Equilibrium Problems

  • Luís Felipe Bueno
  • Gabriel Haeser
  • Felipe Lara
  • Frank Navarro Rojas
    • Categories Nonlinear Optimization

    In this paper, we propose an Augmented Lagrangian algorithm for solving a general class of possible non-convex problems called quasi-equilibrium problems (QEPs). We define an Augmented Lagrangian bifunction associated with QEPs, introduce a secondary QEP as a measure of infeasibility and we discuss several special classes of QEPs within our theoretical framework. For obtaining global … Read more

    Convergence and evaluation-complexity analysis of a regularized tensor-Newton method for solving nonlinear least-squares problems subject to convex constraints

  • Nicholas I. M. Gould
  • Tyrone Rees
  • Jennifer Scott
    • Categories Nonlinear Systems and Least-Squares Tags , , ,

    Given a twice-continuously differentiable vector-valued function $r(x)$, a local minimizer of $\|r(x)\|_2$ within a convex set is sought. We propose and analyse tensor-Newton methods, in which $r(x)$ is replaced locally by its second-order Taylor approximation. Convergence is controlled by regularization of various orders. We establish global convergence to a constrained first-order critical point of $\|r(x)\|_2$, … Read more