An optimal control theory for accelerated optimization
Accelerated optimization algorithms can be generated using a double-integrator model for the search dynamics imbedded in an optimal control problem. CitationunpublishedArticleDownload View PDF
Nonlinear Optimization
Pathfollowing for Parametric Mathematical Programs with Complementarity Constraints
In this paper we study procedures for pathfollowing parametric mathematical pro- grams with complementarity constraints. We present two procedures, one based on the penalty approach to solving standalone MPCCs, and one based on tracing active set bifurcations aris- ing from doubly-active complementarity constraints. We demonstrate the performance of these approaches on a variety of examples … Read more
A two-level distributed algorithm for nonconvex constrained optimization
This paper aims to develop distributed algorithms for nonconvex optimization problems with complicated constraints associated with a network. The network can be a physical one, such as an electric power network, where the constraints are nonlinear power flow equations, or an abstract one that represents constraint couplings between decision variables of different agents. Despite the … Read more
Tangencies and Polynomial Optimization
Given a polynomial function $f \colon \mathbb{R}^n \rightarrow \mathbb{R}$ and a unbounded basic closed semi-algebraic set $S \subset \mathbb{R}^n,$ in this paper we show that the conditions listed below are characterized exactly in terms of the so-called {\em tangency variety} of $f$ on $S$: (i) The $f$ is bounded from below on $S;$ (ii) The … Read more
Inexact restoration with subsampled trust-region methods for finite-sum minimization
Convex and nonconvex finite-sum minimization arises in many scientific computing and machine learning applications. Recently, first-order and second-order methods where objective functions, gradients and Hessians are approximated by randomly sampling components of the sum have received great attention. We propose a new trust-region method which employs suitable approximations of the objective function, gradient and Hessian … Read more
Non-asymptotic Results for Langevin Monte Carlo: Coordinate-wise and Black-box Sampling
Euler-Maruyama and Ozaki discretization of a continuous time diffusion process is a popular technique for sampling, that uses (upto) gradient and Hessian information of the density respectively. The Euler-Maruyama discretization has been used particularly for sampling under the name of Langevin Monte Carlo (LMC) for sampling from strongly log-concave densities. In this work, we make … Read more
Subdifferentials and SNC property of scalarization functionals with uniform level sets and applications
This paper deals with necessary conditions for minimal solutions of constrained and unconstrained optimization problems with respect to general domination sets by using a well-known nonlinear scalarization functional with uniform level sets (called Gerstewitz’ functional in the literature). The primary objective of this work is to establish revised formulas for basic and singular subdifferentials of … Read more
Quasi-Newton Methods for Deep Learning: Forget the Past, Just Sample
We present two sampled quasi-Newton methods: sampled LBFGS and sampled LSR1. Contrary to the classical variants of these methods that sequentially build (inverse) Hessian approximations as the optimization progresses, our proposed methods sample points randomly around the current iterate to produce these approximations. As a result, the approximations constructed make use of more reliable (recent … Read more
Active-set Newton methods and partial smoothness
Diverse optimization algorithms correctly identify, in finite time, intrinsic constraints that must be active at optimality. Analogous behavior extends beyond optimization to systems involving partly smooth operators, and in particular to variational inequalities over partly smooth sets. As in classical nonlinear programming, such active-set structure underlies the design of accelerated local algorithms of Newton type. … Read more
When a maximal angle among cones is nonobtuse
Principal angles between linear subspaces have been studied for their application to statistics, numerical linear algebra, and other areas. In 2005, Iusem and Seeger defined critical angles within a single convex cone as an extension of antipodality in a compact set. Then, in 2016, Seeger and Sossa extended that notion to two cones. This was … Read more