Nonlinear Optimization – Page 140

Iteration-complexity of a Rockafellar’s proximal method of multipliers for convex programming based on second-order approximations

Published: 2016/02/21

This paper studies the iteration-complexity of a new primal-dual algorithm based on Rockafellar’s proximal method of multipliers (PMM) for solving smooth convex programming problems with inequality constraints. In each step, either a step of Rockafellar’s PMM for a second-order model of the problem is computed or a relaxed extragradient step is performed. The resulting algorithm … Read more

A Dual Gradient-Projection Method for Large-Scale Strictly Convex Quadratic Problems

Published: 2016/02/19

Quadratic Programming convex quadratic minimization, gradient projection, sparse computation

The details of a solver for minimizing a strictly convex quadratic objective function subject to general linear constraints is presented. The method uses a gradient projection algorithm enhanced with subspace acceleration to solve the bound-constrained dual optimization problem. Such gradient projection methods are well-known, but are typically employed to solve the primal problem when only … Read more

Numerical Solution of Linear-Quadratic Optimal Control Problems for Switching System

Published: 2016/02/18

Applications - OR and Management Sciences, Constrained Nonlinear Optimization, Control Applications nite approximation, numerical solution, optimal control, switching system

In this paper we obtained an approach to the optimal switching control problem with unknown switching points which it is described in reference [1, 2]. In reference [1], the authors studied the Decomposition of Linear-Quadratic Optimal Control Problems for Two-Steps Systems. In [1], the authors assumed the switching point t1 is xed in the interval … Read more

Gradient Descent only Converges to Minimizers

Published: 2016/02/17

Unconstrained Optimization gradient descent, local minimizer, nonconvex optimization, saddle point problem

We show that gradient descent converges to a local minimizer, almost surely with random initialization. This is proved by applying the Stable Manifold Theorem from dynamical systems theory. ArticleDownload View PDF

On the convergence of stochastic bi-level gradient methods

Published: 2016/02/15

Data-Mining, Nonlinear Optimization bi-level optimization, empirical risk minimization, gradient approximation, stochastic gradient

We analyze the convergence of stochastic gradient methods for bi-level optimization problems. We address two specific cases: first when the outer objective function can be expressed as a finite sum of independent terms, and next when both the outer and inner objective functions can be expressed as finite sums of independent terms. We assume Lipschitz … Read more

A Riemannian rank-adaptive method for low-rank optimization

Published: 2016/01/27, Updated: 2016/02/05

Constrained Nonlinear Optimization, Data-Mining

This paper presents an algorithm that solves optimization problems on a matrix manifold $\mathcal{M} \subseteq \mathbb{R}^{m \times n}$ with an additional rank inequality constraint. The algorithm resorts to well-known Riemannian optimization schemes on fixed-rank manifolds, combined with new mechanisms to increase or decrease the rank. The convergence of the algorithm is analyzed and a weighted … Read more

Constrained Optimization with Low-Rank Tensors and Applications to Parametric Problems with PDEs

Published: 2016/01/25

Optimization of Systems modeled by PDEs, Stochastic Programming, Systems governed by Differential Equations Optimization interior point methods, low-rank tensors, nonlinear optimization, optimal control under uncertainty, parametric variational inequalities, pdes with uncertainties, semismooth newton methods, uncertainty quantification

Low-rank tensor methods provide efficient representations and computations for high-dimensional problems and are able to break the curse of dimensionality when dealing with systems involving multiple parameters. We present algorithms for constrained nonlinear optimization problems that use low-rank tensors and apply them to optimal control of PDEs with uncertain parameters and to parametrized variational inequalities. … Read more

A multiplier method with a class of penalty functions for convex programming

Published: 2016/01/20

Constrained Nonlinear Optimization, Nonlinear Optimization convex optimization, multiplier methods, proximal point algorithm

We consider a class of augmented Lagrangian methods for solving convex programming problems with inequality constraints. This class involves a family of penalty functions and specific values of parameters $p,q,\tilde y \in R$ and $c>0$. The penalty family includes the classical modified barrier and the exponential function. The associated proximal method for solving the dual … Read more

The Legendre Transformation in Modern Optimization

Published: 2016/01/17

Roman Polyak

Constrained Nonlinear Optimization, Convex Optimization dikin's ellipsoid

The Legendre transformation (LET) is a product of a general duality principle: any smooth curve is, on the one hand, a locus of pairs, which satisfy the given equation and, on the other hand, an envelope of a family of its tangent lines. Application of LET to a strictly convex and smooth function leads to … Read more

Multistep stochastic mirror descent for risk-averse convex stochastic programs based on extended polyhedral risk measures

Published: 2016/01/15, Updated: 2016/09/01

Vincent Guigues

Nonlinear Optimization, Stochastic Programming multistep stochastic mirror descent, risk measures, robust stochastic approximation, stochastic optimization

We consider risk-averse convex stochastic programs expressed in terms of extended polyhedral risk measures. We derive computable confidence intervals on the optimal value of such stochastic programs using the Robust Stochastic Approximation and the Stochastic Mirror Descent (SMD) algorithms. When the objective functions are uniformly convex, we also propose a multistep extension of the Stochastic … Read more